Author SHA1 Message Date
Diomidis Spinellis 95cbd302fe Fix handling of lines ending without a newline 2026-08-20 21:05:37 +02:00
Diomidis Spinellis 1e390a63c0 Revert d7a713e fast_io.rs pending_newline addition
It violated layering by having it track higher-level newline behavior
and caused R of non-newline terminated files to fail; (see
read_one_no_newline test).
2026-08-20 21:05:37 +02:00
Diomidis SpinellisandMukunda Katta 452e249296 Implement GNU R (read next line) command
Co-authored-by: Mukunda Katta <mukunda.vjcs6@gmail.com>
2026-08-20 21:05:37 +02:00
Diomidis Spinellis c8b5dc53e3 Fix multiple write commands to same file
Maintining different writers for the same path can fail due to buffering
or underlying OS restrictions. In particular, this fails on Windows.
Fix it through a map of canonicalized write paths.
2026-08-20 10:20:38 +03:00
Diomidis SpinellisandMukunda Katta 714ec8a7bf Implement GNU W (write pattern first line) command
While at it, also fix the w command on patterns missing a newline.

Co-authored-by: Mukunda Katta <mukunda.vjcs6@gmail.com>
2026-08-19 20:22:54 +03:00
Diomidis Spinellis 79e5df0df8 Document F command extension 2026-08-19 14:32:26 +03:00
Diomidis Spinellis 3f72f93c5d Support F (output filename) command (#528)
This also requires storing input_name as PathBuf to avoid lossy
conversions when outputting an OS-specific byte sequence representing
the file name.
2026-08-19 12:20:27 +02:00
renovate[bot] d837ddf398 Update Rust crate similar to v3.2.0 2026-08-17 13:53:54 +02:00
renovate[bot] b1ca27e82d Update dawidd6/action-download-artifact action to v23 2026-08-16 10:06:49 +02:00
renovate[bot] ce711ac531 Update Rust crate clap_mangen to v0.3.3 2026-08-13 08:11:59 +02:00
renovate[bot] 618a3f7c39 Update Rust crate ctor to v1.0.13 2026-08-10 08:16:11 +02:00
Sylvestre Ledru 079eb9340e Add cargo-binstall support (#520)
Point cargo-binstall at the archives published by dist in the GitHub
releases, so 'cargo binstall sed' fetches a prebuilt binary instead of
building from source.
2026-08-09 10:00:46 +02:00
Sylvestre Ledru 2ce633cb8d prepare version 0.2.0 (#519) 2026-08-09 01:57:25 +02:00
Mukunda Rao Katta 91465e566b fix: identify uutils in cli output (#435) 2026-08-09 00:55:37 +02:00
Sylvestre Ledru d842aa305e sed: fix clippy cast_possible_truncation errors (#518) 2026-08-09 00:52:26 +02:00
xtqqczze d28a1f8e73 chore(cargo): remove deprecated package authors field (#444) 2026-08-08 23:28:22 +02:00
renovate[bot] af5f8afdcb Update Rust crate clap_complete to v4.6.9 2026-08-07 08:07:56 +02:00
renovate[bot] 8f3f126816 Update Rust crate clap_mangen to v0.3.2 2026-08-07 08:05:08 +02:00
renovate[bot] c771318a7a Update Rust crate clap to v4.6.6 2026-08-07 08:03:52 +02:00
renovate[bot] 36a1cf4954 Update Rust crate uucore to 0.10.0 2026-08-06 09:02:46 +02:00
renovate[bot] 13bb82ba13 Update Rust crate clap_mangen to v0.3.1 2026-08-06 08:32:10 +02:00
renovate[bot] cb9eb1da42 Update Rust crate uufuzz to 0.10.0 2026-08-06 08:15:55 +02:00
renovate[bot] b101462d1b Update Rust crate uutests to 0.10.0 2026-08-06 08:10:31 +02:00
oech3 9f2e1a1ad4 remove sysinfo dep 2026-08-05 14:18:30 +02:00
Franklin-Qi 2b092d53b0 fast_io.rs: replace the unsafe libc::copy_file_range call by rustix (#458)
Replace the unsafe `libc::copy_file_range` call with the safe `rustix::fs::copy_file_range`.

Closes: #442
2026-08-05 13:32:45 +02:00
renovate[bot] 550d545d72 Update Rust crate similar to v3.1.2 2026-08-05 08:01:27 +02:00
Diomidis Spinellisandmukunda katta a5d219764f feat: add T command (branch on no-substitution) (#508)
* feat: add T command (branch on no-substitution)

Implements the GNU `T LABEL` command: branch to LABEL if no s///
substitution has succeeded since the last input line was read or the
last t/T was taken. It is the inverse of `t` and shares the same
address and label parsing.

Wires 'T' into get_cmd_spec, resolve_branch_targets, and the processor
branch logic, resetting the substitution flag on evaluation just like
't'. Adds integration tests (simple branch + flag-clear) and a cmd-spec
unit test.

Closes #395

Co-authored-by: Diomidis Spinellis <dds@aueb.gr>

* ci: re-trigger checks (windows runner crates.io download flake)

* Disable T in POSIX mode

---------

Co-authored-by: mukunda katta <mukunda.vjcs6@gmail.com>
2026-08-02 17:47:18 +03:00
Develandpre-commit-ci[bot] 397ab86302 Implement v command #397 (#420)
* implement version command

* add tests for version option

* fix patch comparsion

* absurd extreme comparsion fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* accept missing minor or patch

* guard 'v' with if posix

* add a few tests

* add test_default_version

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* check if minor is empty and push zero instead of shadowing

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-08-01 18:18:05 +03:00
32 changed files with 1467 additions and 772 deletions

No files matched your search

+1 -1
View File
@@ -134,7 +134,7 @@ jobs:
persist-credentials: false
- name: Retrieve reference artifacts
uses: dawidd6/action-download-artifact@v21
uses: dawidd6/action-download-artifact@v23
continue-on-error: true
with:
workflow: GnuTests.yml
Generated
+58 -319
View File
File diff suppressed because it is too large. Load diff
+19 -8
View File
@@ -5,8 +5,7 @@
[package]
name = "sed"
version = "0.1.1"
authors = ["uutils developers"]
version = "0.2.0"
license = "MIT"
description = "sed ~ implemented as universal (cross-platform) utils, written in Rust"
default-run = "sed"
@@ -21,6 +20,19 @@ rust-version = "1.88"
build = "build.rs"
# cargo-binstall: fetch the archives published by `dist` in the GitHub releases
# instead of building from source. The layout below has to stay in sync with
# `dist-workspace.toml` (targets) and the archives it produces:
# sed-<target>.tar.xz (zip on Windows), containing a sed-<target>/ directory.
# Note that the release tags carry no `v` prefix, hence `{ version }`.
[package.metadata.binstall]
pkg-url = "{ repo }/releases/download/{ version }/{ name }-{ target }.{ archive-suffix }"
bin-dir = "{ name }-{ target }/{ bin }{ binary-ext }"
pkg-fmt = "txz"
[package.metadata.binstall.overrides.x86_64-pc-windows-msvc]
pkg-fmt = "zip"
[features]
default = ["feat_common_core"]
@@ -29,7 +41,6 @@ sed = []
[workspace.dependencies]
assert_fs = "1.1.3"
bytesize = "2.0.0"
chrono = { version = "0.4.37", default-features = false, features = [
"clock",
] }
@@ -48,11 +59,11 @@ predicates = "3.1.3"
rand = { version = "0.10.0" }
regex = "1.10.4"
sha2 = "0.11"
sysinfo = "0.38"
tempfile = "3.10.1"
terminal_size = "0.4.2"
textwrap = { version = "0.16.1", features = ["terminal_size"] }
uucore = { version = "0.9.0", features = ["libc"] }
uucore = { version = "0.10.0", features = ["libc"] }
rustix = "1.1.4"
xattr = "1.3.1"
@@ -66,11 +77,11 @@ memmap2.workspace = true
phf = { workspace = true }
predicates = { workspace = true }
regex = { workspace = true }
sysinfo = { workspace = true }
tempfile = { workspace = true }
terminal_size = { workspace = true }
textwrap = { workspace = true }
uucore = { workspace = true }
rustix = { workspace = true }
[dev-dependencies]
assert_fs = { workspace = true }
@@ -85,7 +96,7 @@ regex = { workspace = true }
sha2 = { workspace = true }
tempfile = { workspace = true }
uucore = { workspace = true, features = ["entries", "process", "signals", "benchmark"] }
uutests = "0.9.0"
uutests = "0.10.0"
[target.'cfg(unix)'.dev-dependencies]
xattr = { workspace = true }
@@ -135,7 +146,7 @@ must_use_candidate = "allow" # 56
needless_raw_string_hashes = "allow" # 20
needless_pass_by_value = "allow" # 15
missing_panics_doc = "allow" # 12
cast_possible_truncation = "allow" # 7
# cast_possible_truncation = "allow" # 7
unnecessary_wraps = "allow" # 6
match_wildcard_for_single_variants = "allow" # 4
cast_sign_loss = "allow" # 4
+13
View File
@@ -31,6 +31,13 @@ Further work aims to:
We provide a Linux x86_64 binary archive from the main branch at
https://github.com/uutils/sed/releases/tag/latest-commit .
If you have [cargo-binstall](https://github.com/cargo-bins/cargo-binstall),
the released binaries can be installed directly with:
```bash
cargo binstall sed
```
For other platforms, ensure you have Rust installed on your system. You can install Rust through [rustup](https://rustup.rs/).
Clone the repository and build the project using Cargo:
@@ -94,8 +101,14 @@ cargo test
in the specified text.
* The `a`, `i`, `=`, `l`, `q` and `r` commands support address range as an extension to POSIX.
* The substitution command replacement group `\0` is a synonym for &.
* An `F` command outputs the name of the file currently being processed.
* The `p` command adds a missing newline.
(POSIX is silent; FreeBSD sed ignores the newline; original sed ignores
the line).
* A `Q` command (optionally followed by an exit code) quits immediately.
* The `q` command can be optionally followed by an exit code.
* An `R` schedules reading the next line from the specified file.
* A `W` command writes to a file the pattern's first line.
* The `l` command can be optionally followed by the output width.
* The `--follow-symlinks` option for in-place editing.
* The `--sandbox` option that limits potentially destructive commands.
+12 -126
View File
@@ -646,15 +646,6 @@ version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61807f77802ff30975e01f4f071c8ba10c022052f98b3294119f3e615d13e5be"
[[package]]
name = "ntapi"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
dependencies = [
"winapi",
]
[[package]]
name = "num-bigint"
version = "0.4.6"
@@ -683,25 +674,6 @@ dependencies = [
"autocfg",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags",
]
[[package]]
name = "objc2-io-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
dependencies = [
"libc",
"objc2-core-foundation",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -925,7 +897,7 @@ checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "sed"
version = "0.1.1"
version = "0.2.0"
dependencies = [
"clap",
"clap_complete",
@@ -937,7 +909,7 @@ dependencies = [
"phf_codegen",
"predicates",
"regex",
"sysinfo",
"rustix",
"tempfile",
"terminal_size",
"textwrap",
@@ -1013,9 +985,9 @@ checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "similar"
version = "3.1.1"
version = "3.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6505efef05804732ed8a3f2d4f279429eb485bd69d5b0cc6b19cc02005cda16"
checksum = "4f66ca1f7aca2474dc10c942eb22feffc897735f54cd1db90138c2fddb490987"
dependencies = [
"bstr",
]
@@ -1055,20 +1027,6 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "sysinfo"
version = "0.38.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ab6a2f8bfe508deb3c6406578252e491d299cbbf3bc0529ecc3313aee4a52f"
dependencies = [
"libc",
"memchr",
"ntapi",
"objc2-core-foundation",
"objc2-io-kit",
"windows",
]
[[package]]
name = "tempfile"
version = "3.27.0"
@@ -1076,7 +1034,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"getrandom 0.4.2",
"once_cell",
"rustix",
"windows-sys",
@@ -1194,9 +1152,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uucore"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "069b34217c27f611e1589f540f58118dbf226a9e407d38ab472052ff075a1dc2"
checksum = "e97dd0fd8bb2ed12affba9f956c3b94b8ad3f9432746717475420e916ae0b8d4"
dependencies = [
"bigdecimal",
"clap",
@@ -1212,6 +1170,7 @@ dependencies = [
"rustix",
"thiserror",
"unic-langid",
"unicode-width",
"uucore_procs",
"wild",
]
@@ -1233,9 +1192,9 @@ dependencies = [
[[package]]
name = "uucore_procs"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34337da211e7abfff7189b794afb3b5018fe356fb36474b73645458fc1201350"
checksum = "907f1b828046ea42cdb4bd898f05b48ab873a41937bf75ebe12f09a5bf411f58"
dependencies = [
"proc-macro2",
"quote",
@@ -1243,9 +1202,9 @@ dependencies = [
[[package]]
name = "uufuzz"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e4192b67d5e7f0a3300db353ec3217beee89442f02c11d8ae108867eee392d0"
checksum = "72d4c5bac0f8ccabaa8e8ccbf2724446f8b007c06ad343fabfe1a1525a4ff4f2"
dependencies = [
"console",
"rand",
@@ -1361,49 +1320,6 @@ dependencies = [
"glob",
]
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580"
dependencies = [
"windows-collections",
"windows-core",
"windows-future",
"windows-numerics",
]
[[package]]
name = "windows-collections"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610"
dependencies = [
"windows-core",
]
[[package]]
name = "windows-core"
version = "0.62.2"
@@ -1417,17 +1333,6 @@ dependencies = [
"windows-strings",
]
[[package]]
name = "windows-future"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb"
dependencies = [
"windows-core",
"windows-link",
"windows-threading",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
@@ -1456,16 +1361,6 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-numerics"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26"
dependencies = [
"windows-core",
"windows-link",
]
[[package]]
name = "windows-result"
version = "0.4.1"
@@ -1493,15 +1388,6 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-threading"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37"
dependencies = [
"windows-link",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
+2 -2
View File
@@ -14,8 +14,8 @@ libc = "0.2.153"
tempfile = "3.15.0"
rand = { version = "0.10.0" }
similar = "3.0.0"
uucore = { version = "0.9.0", features = ["libc"] }
uufuzz = "0.9.0"
uucore = { version = "0.10.0", features = ["libc"] }
uufuzz = "0.10.0"
sed = { path = ".." }
+19 -14
View File
@@ -10,7 +10,8 @@
use crate::sed::error_handling::{ScriptLocation, runtime_error};
use crate::sed::fast_regex::{Captures, Match, Regex};
use crate::sed::named_writer::NamedWriter;
use crate::sed::named_io::{NamedReader, NamedWriter};
use crate::sed::processor::RecordSeparatorState;
use crate::sed::script_char_provider::ScriptCharProvider;
use crate::sed::script_line_provider::ScriptLineProvider;
@@ -40,8 +41,8 @@ pub struct ProcessingContext {
pub null_data: bool,
// Other context
/// Currently processed input file name (not script) in quoted form
pub input_name: String,
/// Currently processed input file name (not script)
pub input_name: PathBuf,
/// Current input line number
pub line_number: usize,
/// True if this is the last address of a range
@@ -72,13 +73,16 @@ pub struct ProcessingContext {
pub substitution_made: bool,
/// Elements to append at the end of each command processing cycle
pub append_elements: Vec<AppendElement>,
/// State used to separate consecutive output records.
pub rss: RecordSeparatorState,
}
#[derive(Clone, Debug)]
/// Elements that shall be appended at the end of each command processing cycle
pub enum AppendElement {
Text(Rc<[u8]>), // The specified text bytes
Path(PathBuf), // The contents of the specified file path
Text(Rc<[u8]>), // Text from the a command
ReaderText(Rc<[u8]>), // One line from the R command
Path(PathBuf), // The contents of the specified file path
}
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
@@ -255,12 +259,12 @@ impl Default for Transliteration {
/// Create a new Transliteration with identity mapping for the fast-path.
fn default() -> Self {
let mut fast = [0u8; 256];
for (i, slot) in fast.iter_mut().enumerate() {
*slot = i as u8;
for (slot, value) in fast.iter_mut().zip(0..=u8::MAX) {
*slot = value;
}
let mut unicode_fast = ['\0'; COMMON_UNICODE];
for (i, slot) in unicode_fast.iter_mut().enumerate() {
*slot = char::from_u32(i as u32).unwrap_or('\0');
for (slot, cp) in unicode_fast.iter_mut().zip(0u32..) {
*slot = char::from_u32(cp).unwrap_or('\0');
}
Self {
byte_fast: fast,
@@ -361,13 +365,14 @@ impl Command {
#[derive(Debug)]
/// Command-specific data
/// After parsing, t, b Label elements are converted into BranchTarget ones.
/// After parsing, t, T, b Label elements are converted into BranchTarget ones.
pub enum CommandData {
None,
BranchTarget(Option<Rc<RefCell<Command>>>), // Commands for 'b', 't', '{'
Label(Option<String>), // Label name for 'b', 't', ':'
BranchTarget(Option<Rc<RefCell<Command>>>), // Commands for 'b', 't', 'T', '{'
Label(Option<String>), // Label name for 'b', 't', 'T', ':'
Path(PathBuf), // File path for 'r'
NamedWriter(Rc<RefCell<NamedWriter>>), // File output for 'w'
NamedReader(Rc<RefCell<NamedReader>>), // File input for 'R'
NamedWriter(Rc<RefCell<NamedWriter>>), // File output for 'w', 'W'
Number(usize), // Number for 'l', 'q', 'Q' (GNU)
Substitution(Box<Substitution>), // Substitute command 's'
Text(Rc<[u8]>), // Text for 'a', 'c', 'i'
@@ -593,7 +598,7 @@ mod tests {
fn test_all_fast_path_mapped_to_space() {
let mut t = Transliteration::default();
for cp in 0..COMMON_UNICODE {
if let Some(ch) = char::from_u32(cp as u32) {
if let Some(ch) = u32::try_from(cp).ok().and_then(char::from_u32) {
t.insert(ch, ' ');
}
}
+108 -5
View File
@@ -18,7 +18,7 @@ use crate::sed::delimited_parser::{
};
use crate::sed::error_handling::{ScriptLocation, compilation_error, semantic_error};
use crate::sed::fast_regex::Regex;
use crate::sed::named_writer::NamedWriter;
use crate::sed::named_io::{NamedReader, NamedWriter};
use crate::sed::script_char_provider::ScriptCharProvider;
use crate::sed::script_line_provider::{ScriptLineProvider, ScriptValue};
@@ -213,8 +213,8 @@ fn resolve_branch_targets(
resolve_branch_targets(Some(sub_head.clone()), context)?;
}
// Only for 't' or 'b' commands:
if matches!(cmd.code, 't' | 'b') {
// Only for 't', 'T', or 'b' commands:
if matches!(cmd.code, 't' | 'T' | 'b') {
// Take ownership of the current data
let old_data = mem::replace(&mut cmd.data, CommandData::None);
@@ -1107,7 +1107,7 @@ fn compile_empty_command(
Ok(CommandHandling::Continue)
}
// Handles r
// Handles r, R
fn compile_read_file_command(
lines: &mut ScriptLineProvider,
line: &mut ScriptCharProvider,
@@ -1118,7 +1118,11 @@ fn compile_read_file_command(
return compilation_error(lines, line, ERR_SANDBOX);
}
let path = read_file_path(lines, line)?;
cmd.data = CommandData::Path(path);
cmd.data = if cmd.code == 'R' {
CommandData::NamedReader(NamedReader::new(path))
} else {
CommandData::Path(path)
};
Ok(CommandHandling::Continue)
}
@@ -1373,6 +1377,72 @@ fn compile_text_command_posix(
Ok(CommandHandling::Continue)
}
// Handle v
fn compile_version_command(
lines: &mut ScriptLineProvider,
line: &mut ScriptCharProvider,
_cmd: &mut Command,
_context: &mut ProcessingContext,
) -> UResult<CommandHandling> {
// Claim version partify with GNU sed 4.9
const GNU_MAJOR: u8 = 4;
const GNU_MINOR: u8 = 9;
const GNU_PATCH: u8 = 0;
line.advance();
line.eat_spaces(); // Skip any leading whitespace.
let mut major = String::new();
let mut minor = String::new();
let mut patch = String::new();
let mut ver_semantic = 0;
while !line.eol() {
if line.current() == '.' {
ver_semantic += 1;
line.advance();
} else {
match ver_semantic {
0 => major.push(line.current()),
1 => minor.push(line.current()),
2 => patch.push(line.current()),
_ => return compilation_error(lines, line, "invalid version of sed"),
}
line.advance();
}
}
if major.is_empty() {
major = GNU_MAJOR.to_string();
minor = GNU_MINOR.to_string();
patch = GNU_PATCH.to_string();
}
if minor.is_empty() {
minor.push('0');
}
if patch.is_empty() {
patch.push('0');
}
match major.parse::<u8>() {
Ok(major_int) => match minor.parse::<u8>() {
Ok(minor_int) => match patch.parse::<u8>() {
Ok(patch_int) => {
if minor_int <= GNU_MINOR && major_int <= GNU_MAJOR && patch_int == GNU_PATCH {
return Ok(CommandHandling::Continue);
}
compilation_error(lines, line, "expected newer version of sed")
}
Err(_) => compilation_error(lines, line, "invalid version of sed"),
},
Err(_) => compilation_error(lines, line, "invalid version of sed"),
},
Err(_) => compilation_error(lines, line, "invalid version of sed"),
}
}
// Handles e
// With no argument, the command executes the pattern space as a shell
// command at runtime. With an argument, the rest of the line is the
@@ -1566,6 +1636,16 @@ fn get_cmd_spec(
n_addr: 2,
handler: compile_execute_command,
}),
// F is a GNU extension
'F' if !posix => Ok(CommandSpec {
n_addr: 2,
handler: compile_empty_command,
}),
// R is a GNU extension
'R' if !posix => Ok(CommandSpec {
n_addr: 2,
handler: compile_read_file_command,
}),
'r' => Ok(CommandSpec {
n_addr: if posix { 1 } else { 2 },
handler: compile_read_file_command,
@@ -1574,6 +1654,15 @@ fn get_cmd_spec(
n_addr: 2,
handler: compile_subst_command,
}),
'T' if !posix => Ok(CommandSpec {
n_addr: 2,
handler: compile_label_command,
}),
// W is a GNU extension
'W' if !posix => Ok(CommandSpec {
n_addr: 2,
handler: compile_write_file_command,
}),
'w' => Ok(CommandSpec {
n_addr: 2,
handler: compile_write_file_command,
@@ -1582,6 +1671,10 @@ fn get_cmd_spec(
n_addr: 2,
handler: compile_trans_command,
}),
'v' if !posix => Ok(CommandSpec {
n_addr: 0,
handler: compile_version_command,
}),
_ => compilation_error(lines, line, format!("invalid command code `{cmd_code}'")),
}
}
@@ -1665,6 +1758,16 @@ mod tests {
);
}
#[test]
fn test_lookup_branch_commands() {
// b, t, and T all share compile_label_command and accept 2 addresses.
for code in ['b', 't', 'T'] {
let (lines, line) = make_providers("123abc");
let cmd = get_cmd_spec(&lines, &line, code, false).unwrap();
assert_eq!(cmd.n_addr, 2, "command `{code}` should accept 2 addresses");
}
}
// Utility to create a ScriptCharProvider from a &str
fn char_provider_from(s: &str) -> ScriptCharProvider {
ScriptCharProvider::new(s)
+2 -1
View File
@@ -14,6 +14,7 @@ use crate::sed::script_line_provider::ScriptLineProvider;
use std::rc::Rc;
use uucore::display::Quotable;
use uucore::error::{UResult, USimpleError};
#[derive(Clone, Debug)]
@@ -108,7 +109,7 @@ pub fn input_runtime_error<T>(
location.input_name,
location.line_number,
location.column_number,
context.input_name,
context.input_name.quote(),
context.line_number,
msg.to_string()
),
+157 -140
View File
@@ -29,6 +29,11 @@ use std::marker::PhantomData;
#[cfg(unix)]
use std::os::fd::RawFd;
#[cfg(unix)]
use rustix::fd::BorrowedFd;
#[cfg(target_os = "linux")]
use rustix::fs::copy_file_range as rustix_copy_file_range;
#[cfg(unix)]
use std::os::unix::io::AsRawFd;
@@ -402,9 +407,24 @@ impl FastCopy {
Self {
fd,
is_regular: ftype as libc::mode_t == libc::S_IFREG,
block_size: st.st_blksize as usize,
// st_blksize is always positive and small; fall back to a common
// block size if it ever doesn't fit in usize.
block_size: usize::try_from(st.st_blksize).unwrap_or(4096),
}
}
/// Return a borrowed file descriptor.
///
/// # Safety invariant
///
/// `FastCopy` only stores file descriptors obtained from open
/// [`File`](std::fs::File) objects. Those `File`s are kept alive
/// by the owning [`MmapLineCursor`] / output-buffer, guaranteeing
/// the fd remains valid for the duration of any borrow.
pub fn as_fd(&self) -> BorrowedFd<'_> {
// SAFETY: self.fd is a valid file descriptor owned by a live File.
unsafe { BorrowedFd::borrow_raw(self.fd) }
}
}
#[cfg(test)]
@@ -572,11 +592,8 @@ pub struct OutputBuffer {
max_pending_write: usize, // Max bytes to keep before flushing
#[cfg(unix)]
mmap_chunk: Option<MmapOutput>, // Chunk to write
// True when the last write didn't end with \n; the \n is deferred so
// that commands like `p` don't emit a spurious newline under -n.
pending_newline: bool,
#[cfg(test)]
low_level_flushes: usize, // Number of system call flushes
low_level_flushes: usize, // Number of system call flushes
}
/// Threshold to use buffered writes for output
@@ -607,7 +624,6 @@ impl OutputBuffer {
pub fn new(w: Box<dyn OutputWrite + 'static>) -> Self {
Self {
out: BufWriter::new(w),
pending_newline: false,
#[cfg(test)]
low_level_flushes: 0,
}
@@ -626,7 +642,6 @@ impl OutputBuffer {
fast_copy,
max_pending_write,
mmap_chunk: None,
pending_newline: false,
#[cfg(test)]
low_level_flushes: 0,
}
@@ -705,12 +720,6 @@ impl OutputBuffer {
return Ok(());
}
if self.pending_newline {
self.flush_mmap(WriteRange::Complete)?;
self.out.write_all(b"\n")?;
self.pending_newline = false;
}
match &new_chunk.content {
IOChunkContent::MmapInput {
full_span,
@@ -756,7 +765,6 @@ impl OutputBuffer {
len: new_len,
});
}
self.pending_newline = !new_chunk.is_newline_terminated();
}
IOChunkContent::Owned {
@@ -769,7 +777,6 @@ impl OutputBuffer {
if *has_newline {
self.out.write_all(b"\n")?;
}
self.pending_newline = !has_newline;
}
}
Ok(())
@@ -798,10 +805,10 @@ impl OutputBuffer {
if chunk.in_fast_copy.is_regular && self.fast_copy.is_regular {
portable_copy_file_range(
chunk.out_ptr,
chunk.in_fast_copy.fd,
chunk.in_fast_copy.as_fd(),
// Input file offset
unsafe { chunk.out_ptr.offset_from(chunk.base_ptr) } as libc::off_t,
self.fast_copy.fd,
self.fast_copy.as_fd(),
chunk.len,
// Alignment block size: the largest of the two
chunk.in_fast_copy.block_size.max(self.fast_copy.block_size),
@@ -818,16 +825,6 @@ impl OutputBuffer {
Ok(())
}
/// Write a deferred newline if the last output didn't end with one.
pub fn flush_pending_newline(&mut self) -> io::Result<()> {
if self.pending_newline {
self.flush_mmap(WriteRange::Complete)?;
self.out.write_all(b"\n")?;
self.pending_newline = false;
}
Ok(())
}
/// Flush everything: pending mmap and buffered data.
pub fn flush(&mut self) -> io::Result<()> {
self.flush_mmap(WriteRange::Complete)?; // flush mmap if any
@@ -843,11 +840,6 @@ impl OutputBuffer {
return Ok(());
}
if self.pending_newline {
self.out.write_all(b"\n")?;
self.pending_newline = false;
}
match &chunk.content {
IOChunkContent::Owned {
content,
@@ -858,21 +850,11 @@ impl OutputBuffer {
if *has_newline {
self.out.write_all(b"\n")?;
}
self.pending_newline = !has_newline;
Ok(())
}
}
}
/// Write a deferred newline if the last output didn't end with one.
pub fn flush_pending_newline(&mut self) -> io::Result<()> {
if self.pending_newline {
self.out.write_all(b"\n")?;
self.pending_newline = false;
}
Ok(())
}
/// Flush everything: pending mmap and buffered data.
pub fn flush(&mut self) -> io::Result<()> {
self.out.flush() // then flush buffered data
@@ -916,9 +898,9 @@ fn reliable_write(fd: i32, ptr: *const u8, len: usize) -> std::io::Result<usize>
#[allow(unused_variables)]
fn portable_copy_file_range(
in_ptr: *const u8,
in_fd: i32,
in_fd: BorrowedFd<'_>,
in_off: libc::off_t,
out_fd: i32,
out_fd: BorrowedFd<'_>,
len: usize,
block_size: usize,
cover: WriteRange,
@@ -933,7 +915,7 @@ fn portable_copy_file_range(
}
#[cfg(not(all(target_os = "linux", target_env = "gnu")))]
{
reliable_write(out_fd, in_ptr, len)
reliable_write(out_fd.as_raw_fd(), in_ptr, len)
}
}
@@ -944,37 +926,34 @@ fn portable_copy_file_range(
#[cfg(all(target_os = "linux", target_env = "gnu"))]
fn reliable_copy_file_range(
in_ptr: *const u8,
in_fd: i32,
in_fd: BorrowedFd<'_>,
mut in_off: libc::off_t,
out_fd: i32,
out_fd: BorrowedFd<'_>,
len: usize,
) -> std::io::Result<usize> {
let mut pending = len;
while pending > 0 {
let ret = unsafe {
libc::copy_file_range(
in_fd,
&raw mut in_off,
out_fd,
std::ptr::null_mut(), // Use and update output offset
pending,
0,
)
};
if ret < 0 {
let err = io::Error::last_os_error();
return match err.raw_os_error() {
Some(libc::ENOSYS) | Some(libc::EOPNOTSUPP) | Some(libc::EXDEV) => {
// Fallback to write(2).
reliable_write(out_fd, in_ptr, pending)
}
_ => Err(err),
};
} else if ret == 0 {
// EOF reached
break;
let mut in_off_u64 = in_off as u64;
let result: std::io::Result<usize> =
rustix_copy_file_range(in_fd, Some(&mut in_off_u64), out_fd, None, pending)
.map_err(std::io::Error::from);
match result {
Ok(0) => break,
Ok(ret) => {
pending -= ret;
in_off = in_off_u64 as libc::off_t;
}
Err(err) => {
return match err.raw_os_error() {
Some(libc::ENOSYS) | Some(libc::EOPNOTSUPP) | Some(libc::EXDEV) => {
// Fallback to write(2).
reliable_write(out_fd.as_raw_fd(), in_ptr, pending)
}
_ => Err(err),
};
}
}
pending -= ret as usize;
}
Ok(len)
}
@@ -986,24 +965,27 @@ fn reliable_copy_file_range(
#[cfg(all(target_os = "linux", target_env = "gnu"))]
fn aligned_copy_file_range(
mut in_ptr: *const u8,
in_fd: i32,
in_fd: BorrowedFd<'_>,
mut in_off: libc::off_t,
out_fd: i32,
out_fd: BorrowedFd<'_>,
len: usize,
block_size: usize,
cover: WriteRange,
) -> std::io::Result<usize> {
// 1. Get current output offset.
let res = unsafe { libc::lseek(out_fd, 0, libc::SEEK_CUR) as i64 };
if res < 0 {
return Err(std::io::Error::last_os_error());
}
let out_off = res as usize;
// Get current output offset.
// If either offset doesn't fit in usize (only possible on 32-bit
// targets), give up on alignment and just copy everything.
let (Ok(out_off), Ok(in_off_bytes)) = (
usize::try_from(rustix::fs::tell(out_fd)?),
usize::try_from(in_off),
) else {
return reliable_copy_file_range(in_ptr, in_fd, in_off, out_fd, len);
};
let mut pending = len;
// Obtain head alignment.
// Bytes to end of block:
let remainder = in_off as usize % block_size;
let remainder = in_off_bytes % block_size;
// Bytes to write (block_size becomes 0):
let head_align = (block_size - remainder) % block_size;
@@ -1015,7 +997,7 @@ fn aligned_copy_file_range(
if head_align > 0 {
// Align the two files on a block boundary.
let head_len = head_align.min(pending);
reliable_write(out_fd, in_ptr, head_len)?;
reliable_write(out_fd.as_raw_fd(), in_ptr, head_len)?;
in_ptr = unsafe { in_ptr.add(head_len) };
in_off += head_len as i64;
pending -= head_len;
@@ -1029,7 +1011,7 @@ fn aligned_copy_file_range(
// Copy tail if needed.
if pending > 0 && cover == WriteRange::Complete {
in_ptr = unsafe { in_ptr.add(aligned_len) };
pending -= reliable_write(out_fd, in_ptr, pending)?;
pending -= reliable_write(out_fd.as_raw_fd(), in_ptr, pending)?;
}
Ok(len - pending)
@@ -1062,6 +1044,8 @@ mod tests {
#[cfg(all(target_os = "linux", target_env = "gnu"))]
use std::io::{self, Write};
use std::io::{Seek, SeekFrom};
#[cfg(target_os = "linux")]
use std::os::unix::io::AsFd;
use tempfile::NamedTempFile;
use tempfile::tempfile;
@@ -1793,8 +1777,8 @@ mod tests {
infile.write_all(data).unwrap();
infile.rewind().unwrap();
let in_fd = infile.as_raw_fd();
let out_fd = outfile.as_raw_fd();
let in_fd = infile.as_fd();
let out_fd = outfile.as_fd();
let in_ptr = data.as_ptr();
// Copy with block size 4, cover = Complete
@@ -1829,8 +1813,8 @@ mod tests {
infile.write_all(data).unwrap();
infile.rewind().unwrap();
let in_fd = infile.as_raw_fd();
let out_fd = outfile.as_raw_fd();
let in_fd = infile.as_fd();
let out_fd = outfile.as_fd();
let in_ptr = data.as_ptr();
// Copy with block size 4, cover = Blocks
@@ -1848,6 +1832,94 @@ mod tests {
assert_eq!(&buf, b"abcdefgh");
}
#[test]
#[cfg(target_os = "linux")]
fn test_aligned_copy_respects_existing_out_offset() {
let mut infile = tempfile().unwrap();
let mut outfile = tempfile().unwrap();
// Write a prefix into the output file so tell() returns a non-zero offset.
let prefix = b"HEADER";
outfile.write_all(prefix).unwrap();
outfile.flush().unwrap();
// Input data = 12 bytes (3 blocks of 4 when block_size=4).
let data = b"abcdefghijkl";
infile.write_all(data).unwrap();
infile.rewind().unwrap();
let in_fd = infile.as_fd();
let out_fd = outfile.as_fd();
let in_ptr = data.as_ptr();
let copied = aligned_copy_file_range(
in_ptr,
in_fd,
0,
out_fd,
data.len(),
4,
WriteRange::Complete,
)
.unwrap();
assert_eq!(copied, data.len());
outfile.rewind().unwrap();
let mut buf = Vec::new();
outfile.read_to_end(&mut buf).unwrap();
assert_eq!(&buf, b"HEADERabcdefghijkl");
}
#[test]
#[cfg(target_os = "linux")]
fn test_reliable_copy_file_range_rustix_path() {
let mut infile = tempfile().unwrap();
let mut outfile = tempfile().unwrap();
let data = b"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
infile.write_all(data).unwrap();
infile.rewind().unwrap();
let in_fd = infile.as_fd();
let out_fd = outfile.as_fd();
let in_ptr = data.as_ptr();
// Copy a subrange starting at offset 10.
let copied = reliable_copy_file_range(in_ptr, in_fd, 10, out_fd, 16).unwrap();
assert_eq!(copied, 16);
outfile.rewind().unwrap();
let mut buf = Vec::new();
outfile.read_to_end(&mut buf).unwrap();
assert_eq!(&buf, b"ABCDEFGHIJKLMNOP");
// Now verify that copying into a file with existing data appends correctly.
let mut infile2 = tempfile().unwrap();
let mut outfile2 = tempfile().unwrap();
let input_data = b"abcdefghijklmnopqrstuvwxyz";
infile2.write_all(input_data).unwrap();
infile2.rewind().unwrap();
outfile2.write_all(b"PRE:").unwrap();
outfile2.flush().unwrap();
let in_fd2 = infile2.as_fd();
let out_fd2 = outfile2.as_fd();
let in_ptr2 = input_data.as_ptr();
let copied2 = reliable_copy_file_range(in_ptr2, in_fd2, 5, out_fd2, 10).unwrap();
assert_eq!(copied2, 10);
outfile2.rewind().unwrap();
let mut buf2 = Vec::new();
outfile2.read_to_end(&mut buf2).unwrap();
assert_eq!(&buf2, b"PRE:fghijklmno");
}
///////////////////////////////
// Unit tests for write_chunk()
///////////////////////////////
@@ -1866,7 +1938,6 @@ mod tests {
max_pending_write: 8,
#[cfg(unix)]
mmap_chunk: None,
pending_newline: false,
low_level_flushes: 0,
};
(buf, file)
@@ -1987,58 +2058,4 @@ mod tests {
assert_eq!(out, "world\n");
}
// pending_newline is injected between two no-newline chunks
#[test]
fn pending_newline_injected_between_chunks() {
let (mut buf, mut file) = new_for_test();
buf.write_chunk(&make_owned_chunk("first", false)).unwrap();
buf.write_chunk(&make_owned_chunk("second", true)).unwrap();
buf.out.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut out = String::new();
file.read_to_string(&mut out).unwrap();
assert_eq!(out, "first\nsecond\n");
}
// flush_pending_newline emits the deferred newline
#[test]
fn flush_pending_newline_emits_newline() {
let (mut buf, mut file) = new_for_test();
buf.write_chunk(&make_owned_chunk("foo", false)).unwrap();
assert!(buf.pending_newline);
buf.flush_pending_newline().unwrap();
assert!(!buf.pending_newline);
buf.out.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut out = String::new();
file.read_to_string(&mut out).unwrap();
assert_eq!(out, "foo\n");
}
// write_str strips trailing newline and sets pending_newline correctly
#[test]
fn write_str_with_trailing_newline() {
let (mut buf, mut file) = new_for_test();
buf.write_str("bar\n").unwrap();
assert!(!buf.pending_newline);
buf.out.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut out = String::new();
file.read_to_string(&mut out).unwrap();
assert_eq!(out, "bar\n");
}
#[test]
fn write_str_without_trailing_newline() {
let (mut buf, mut file) = new_for_test();
buf.write_str("baz").unwrap();
assert!(buf.pending_newline);
buf.flush_pending_newline().unwrap();
buf.out.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut out = String::new();
file.read_to_string(&mut out).unwrap();
assert_eq!(out, "baz\n");
}
}
+7 -5
View File
@@ -15,7 +15,7 @@ pub mod error_handling;
pub mod fast_io;
pub mod fast_regex;
pub mod in_place;
pub mod named_writer;
pub mod named_io;
pub mod processor;
pub mod script_char_provider;
pub mod script_line_provider;
@@ -24,15 +24,16 @@ use crate::sed::command::{ByteSpace, CharacterMode, ProcessingContext};
use crate::sed::compiler::compile;
use crate::sed::processor::process_all_files;
use crate::sed::script_line_provider::ScriptValue;
use clap::{Arg, ArgMatches, Command, arg, crate_version};
use clap::{Arg, ArgMatches, Command, arg};
use std::collections::HashMap;
use std::env;
use std::path::PathBuf;
use uucore::error::{UResult, USimpleError, UUsageError};
use uucore::format_usage;
const ABOUT: &str = "Stream editor for filtering and transforming text";
const ABOUT: &str = "Stream editor for filtering and transforming text (part of uutils)";
const USAGE: &str = "sed [OPTION]... [script] [file]...";
const VERSION: &str = concat!(env!("CARGO_PKG_VERSION"), " (uutils)");
#[uucore::main]
pub fn uumain(args: impl uucore::Args) -> UResult<()> {
@@ -62,7 +63,7 @@ pub fn uu_app() -> Command {
let util_name = uucore::util_name();
Command::new(util_name)
.version(crate_version!())
.version(VERSION)
.about(ABOUT)
.override_usage(format_usage(USAGE))
.args_override_self(true)
@@ -247,7 +248,7 @@ fn build_context(matches: &ArgMatches) -> UResult<ProcessingContext> {
character_mode: character_mode_for_locale(&locale)?,
// Other context
input_name: "<stdin>".to_string(),
input_name: PathBuf::from("-"),
line_number: 0,
last_address: false,
last_line: false,
@@ -264,6 +265,7 @@ fn build_context(matches: &ArgMatches) -> UResult<ProcessingContext> {
range_commands: Vec::new(),
substitution_made: false,
append_elements: Vec::new(),
rss: processor::RecordSeparatorState::default(),
})
}
+324
View File
@@ -0,0 +1,324 @@
// An abstraction for output files created on entry and flushed on exit
//
// SPDX-License-Identifier: MIT
// Copyright (c) 2025 Diomidis Spinellis
//
// This file is part of the uutils sed package.
// It is licensed under the MIT License.
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use crate::sed::error_handling::{ScriptLocation, runtime_error};
use std::cell::RefCell;
use std::collections::HashMap;
use std::fs::{self, File, OpenOptions};
use std::io::{BufRead, BufReader, BufWriter, Write};
use std::path::{Path, PathBuf};
use std::rc::Rc;
use uucore::display::Quotable;
use uucore::error::UResult;
thread_local! {
/// Readers indexed by canonical input path, used to share state
/// between same-named path reads.
static READERS: RefCell<HashMap<PathBuf, Rc<RefCell<NamedReader>>>> = RefCell::new(HashMap::new());
/// Writers indexed by canonical output path, used to share same-named
/// paths and to flush writes.
static WRITERS: RefCell<HashMap<PathBuf, Rc<RefCell<NamedWriter>>>> = RefCell::new(HashMap::new());
}
#[derive(Debug)]
/// Reader that shares line-by-line state for GNU sed's R command.
pub struct NamedReader {
path: PathBuf,
reader: Option<BufReader<File>>,
done: bool,
}
impl NamedReader {
/// Create or retrieve the reader associated with `path`.
pub fn new(path: PathBuf) -> Rc<RefCell<Self>> {
let canonical_path = fs::canonicalize(&path).unwrap_or(path);
READERS.with(|readers| {
readers
.borrow_mut()
.entry(canonical_path.clone())
.or_insert_with(|| {
Rc::new(RefCell::new(Self {
path: canonical_path,
reader: None,
done: false,
}))
})
.clone()
})
}
/// Return the path associated with this reader.
pub fn original_path(&self) -> &Path {
&self.path
}
/// Read the next line, including its newline. Missing files and read errors
/// are treated as end-of-file, as required by the R command.
pub fn read_line(&mut self) -> Option<Vec<u8>> {
if self.done {
return None;
}
if self.reader.is_none() {
if let Ok(file) = File::open(&self.path) {
self.reader = Some(BufReader::new(file));
} else {
self.done = true;
return None;
}
}
let mut line = Vec::new();
match self.reader.as_mut().unwrap().read_until(b'\n', &mut line) {
Ok(0) | Err(_) => {
self.done = true;
None
}
Ok(_) => Some(line),
}
}
}
#[derive(Debug)]
/// Writer that tracks its file name for better error messages
pub struct NamedWriter {
pub path: PathBuf,
writer: BufWriter<File>,
location: ScriptLocation,
}
impl NamedWriter {
/// Create a new writer, truncate the file, and register it for flushing.
pub fn new(path: PathBuf, location: ScriptLocation) -> UResult<Rc<RefCell<Self>>> {
let canonical_path = canonicalize_output_path(&path, &location)?;
if let Some(writer) = WRITERS.with(|writers| writers.borrow().get(&canonical_path).cloned())
{
return Ok(writer);
}
let file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&path)
.map_err(|e| {
runtime_error::<()>(&location, format!("creating file {}: {}", path.quote(), e))
.unwrap_err()
})?;
let writer = Rc::new(RefCell::new(NamedWriter {
path,
writer: BufWriter::new(file),
location,
}));
WRITERS.with(|writers| {
writers
.borrow_mut()
.insert(canonical_path, Rc::clone(&writer));
});
Ok(writer)
}
/// Return the path used when this writer was first opened.
pub fn original_path(&self) -> &Path {
&self.path
}
/// Write String to the file, possibly with a newline, returning errors.
pub fn write_line(&mut self, line: &str, newline: bool) -> UResult<()> {
self.write_line_bytes(line.as_bytes(), newline)
}
/// Write bytes to the file, possibly with a newline, returning errors.
pub fn write_line_bytes(&mut self, line: &[u8], newline: bool) -> UResult<()> {
self.writer
.write_all(line)
.and_then(|()| {
if newline {
self.writer.write_all(b"\n")
} else {
Ok(())
}
})
.map_err(|e| {
runtime_error::<()>(
&self.location,
format!("writing to file {}: {e}", self.path.quote()),
)
.unwrap_err()
})
}
/// Flush the writer, returning a descriptive error.
pub fn flush(&mut self) -> UResult<()> {
self.writer.flush().map_err(|e| {
runtime_error::<()>(
&self.location,
format!("writing to file {}: {}", self.path.quote(), e),
)
.unwrap_err()
})
}
}
fn canonicalize_output_path(path: &Path, location: &ScriptLocation) -> UResult<PathBuf> {
let parent = path
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or_else(|| Path::new("."));
let canonical_parent = fs::canonicalize(parent).map_err(|e| {
runtime_error::<()>(location, format!("creating file {}: {}", path.quote(), e)).unwrap_err()
})?;
Ok(match path.file_name() {
Some(file_name) => canonical_parent.join(file_name),
None => canonical_parent,
})
}
/// Flush buffered content to all open files, returning descriptive errors.
pub fn flush_all() -> UResult<()> {
WRITERS.with(|writers| {
for writer in writers.borrow().values() {
writer.borrow_mut().flush()?;
}
Ok(())
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use tempfile::{NamedTempFile, tempdir};
#[test]
fn test_reader_reads_lines_as_bytes() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
fs::write(&path, b"first\nsecond\xE9").unwrap();
let reader = NamedReader::new(path);
assert_eq!(reader.borrow_mut().read_line(), Some(b"first\n".to_vec()));
assert_eq!(
reader.borrow_mut().read_line(),
Some(b"second\xE9".to_vec())
);
assert_eq!(reader.borrow_mut().read_line(), None);
assert_eq!(reader.borrow_mut().read_line(), None);
}
#[test]
fn test_new_reuses_reader_and_shared_position_for_same_path() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
fs::write(&path, b"first\nsecond\n").unwrap();
let first = NamedReader::new(path.clone());
let second = NamedReader::new(path);
assert!(Rc::ptr_eq(&first, &second));
assert_eq!(first.borrow_mut().read_line(), Some(b"first\n".to_vec()));
assert_eq!(second.borrow_mut().read_line(), Some(b"second\n".to_vec()));
}
#[test]
fn test_new_reuses_reader_for_canonical_duplicate_path() {
let dir = tempdir().unwrap();
let path = dir.path().join("input");
fs::write(&path, b"first\nsecond\n").unwrap();
let duplicate_path = dir.path().join(".").join("input");
let first = NamedReader::new(path.clone());
let second = NamedReader::new(duplicate_path);
assert!(Rc::ptr_eq(&first, &second));
assert_eq!(
first.borrow().original_path(),
fs::canonicalize(path).unwrap()
);
assert_eq!(first.borrow_mut().read_line(), Some(b"first\n".to_vec()));
assert_eq!(second.borrow_mut().read_line(), Some(b"second\n".to_vec()));
}
#[test]
fn test_reader_silently_ignores_missing_file() {
let dir = tempdir().unwrap();
let reader = NamedReader::new(dir.path().join("missing"));
assert_eq!(reader.borrow_mut().read_line(), None);
assert_eq!(reader.borrow_mut().read_line(), None);
}
#[test]
fn test_write_line_bytes_appends_newline() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
let writer = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
writer
.borrow_mut()
.write_line_bytes(b"a\xE9", true)
.unwrap();
writer.borrow_mut().flush().unwrap();
assert_eq!(fs::read(path).unwrap(), b"a\xE9\n");
}
#[test]
fn test_write_line_bytes_appends_no_newline() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
let writer = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
writer
.borrow_mut()
.write_line_bytes(b"a\xE9", false)
.unwrap();
writer.borrow_mut().flush().unwrap();
assert_eq!(fs::read(path).unwrap(), b"a\xE9");
}
#[test]
fn test_new_reuses_writer_for_same_path() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
let first = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
let second = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
assert!(Rc::ptr_eq(&first, &second));
assert_eq!(first.borrow().original_path(), path.as_path());
first.borrow_mut().write_line("first", true).unwrap();
second.borrow_mut().write_line("second", true).unwrap();
first.borrow_mut().flush().unwrap();
assert_eq!(fs::read_to_string(path).unwrap(), "first\nsecond\n");
}
#[test]
fn test_new_reuses_writer_for_canonical_duplicate_path() {
let dir = tempdir().unwrap();
let subdir = dir.path().join("subdir");
fs::create_dir(&subdir).unwrap();
let path = subdir.join("output");
let duplicate_path = subdir.join(".").join("output");
let first = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
let second = NamedWriter::new(duplicate_path, ScriptLocation::default()).unwrap();
assert!(Rc::ptr_eq(&first, &second));
assert_eq!(first.borrow().original_path(), path.as_path());
}
}
-117
View File
@@ -1,117 +0,0 @@
// An abstraction for output files created on entry and flushed on exit
//
// SPDX-License-Identifier: MIT
// Copyright (c) 2025 Diomidis Spinellis
//
// This file is part of the uutils sed package.
// It is licensed under the MIT License.
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use crate::sed::error_handling::{ScriptLocation, runtime_error};
use std::cell::RefCell;
use std::fs::{File, OpenOptions};
use std::io::{BufWriter, Write};
use std::path::PathBuf;
use std::rc::Rc;
use uucore::display::Quotable;
use uucore::error::UResult;
thread_local! {
/// Global list of all writers that should be flushed at shutdown
static FLUSH_LIST: RefCell<Vec<Rc<RefCell<NamedWriter>>>> = const { RefCell::new(Vec::new()) };
}
#[derive(Debug)]
/// Writer that tracks its file name for better error messages
pub struct NamedWriter {
pub path: PathBuf,
writer: BufWriter<File>,
location: ScriptLocation,
}
impl NamedWriter {
/// Create a new writer, truncate the file, and register it for flushing.
pub fn new(path: PathBuf, location: ScriptLocation) -> UResult<Rc<RefCell<Self>>> {
let file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&path)
.map_err(|e| {
runtime_error::<()>(&location, format!("creating file {}: {}", path.quote(), e))
.unwrap_err()
})?;
let writer = Rc::new(RefCell::new(NamedWriter {
path,
writer: BufWriter::new(file),
location,
}));
FLUSH_LIST.with(|list| list.borrow_mut().push(Rc::clone(&writer)));
Ok(writer)
}
/// Write a line to the file with a newline, returning descriptive errors.
pub fn write_line(&mut self, line: &str) -> UResult<()> {
self.write_line_bytes(line.as_bytes())
}
/// Write bytes to the file with a newline, returning descriptive errors.
pub fn write_line_bytes(&mut self, line: &[u8]) -> UResult<()> {
self.writer
.write_all(line)
.and_then(|()| self.writer.write_all(b"\n"))
.map_err(|e| {
runtime_error::<()>(
&self.location,
format!("writing to file {}: {e}", self.path.quote()),
)
.unwrap_err()
})
}
/// Flush the writer, returning a descriptive error.
pub fn flush(&mut self) -> UResult<()> {
self.writer.flush().map_err(|e| {
runtime_error::<()>(
&self.location,
format!("writing to file {}: {}", self.path.quote(), e),
)
.unwrap_err()
})
}
}
/// Flush buffered content to the file, returning descriptive errors.
pub fn flush_all() -> UResult<()> {
FLUSH_LIST.with(|cell| {
for handle in cell.borrow().iter() {
handle.borrow_mut().flush()?;
}
Ok(())
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use tempfile::NamedTempFile;
#[test]
fn test_write_line_bytes_appends_newline() {
let file = NamedTempFile::new().unwrap();
let path = file.path().to_path_buf();
let writer = NamedWriter::new(path.clone(), ScriptLocation::default()).unwrap();
writer.borrow_mut().write_line_bytes(b"a\xE9").unwrap();
writer.borrow_mut().flush().unwrap();
assert_eq!(fs::read(path).unwrap(), b"a\xE9\n");
}
}
+225 -30
View File
@@ -17,7 +17,7 @@ use crate::sed::error_handling::{ScriptLocation, input_runtime_error};
use crate::sed::fast_io::{IOChunk, LineReader, OutputBuffer};
use crate::sed::fast_regex::Regex;
use crate::sed::in_place::InPlace;
use crate::sed::named_writer;
use crate::sed::named_io;
use memchr::memchr;
use std::borrow::Cow;
@@ -29,6 +29,40 @@ use std::rc::Rc;
use uucore::display::Quotable;
use uucore::error::{FromIo, UResult, set_exit_code};
#[derive(Clone, Copy, Debug)]
/// Track whether a separator is required before the next output record.
// This implements the following documented GNU sed behavior:
// "If sed prints a line without the terminating newline, it will
// nevertheless print the missing newline as soon as more text is
// sent to the same output stream."
pub struct RecordSeparatorState {
previous_had_newline: bool,
}
impl Default for RecordSeparatorState {
fn default() -> Self {
Self {
previous_had_newline: true,
}
}
}
impl RecordSeparatorState {
/// Begin an output record, terminating the previous one when necessary.
pub fn new_record(&mut self, output: &mut OutputBuffer) -> io::Result<()> {
if !self.previous_had_newline {
output.write_bytes(b"\n")?;
self.previous_had_newline = true;
}
Ok(())
}
/// Record whether the output record ended with a newline.
pub fn has_newline(&mut self, has_newline: bool) {
self.previous_had_newline = has_newline;
}
}
/// Return the specified command variant or panic.
// Example: let path = extract_variant!(command, Path);
macro_rules! extract_variant {
@@ -164,13 +198,41 @@ fn applies(
}
}
/// Write the specified chunk to the output for a given processing context.
fn write_chunk(
/// Output the specified record as a chunk.
fn write_chunk_record(
output: &mut OutputBuffer,
context: &ProcessingContext,
context: &mut ProcessingContext,
chunk: &IOChunk,
) -> std::io::Result<()> {
// Completely empty chunks are no-ops regarding state.
if chunk.is_empty() && !chunk.is_newline_terminated() {
return Ok(());
}
context.rss.new_record(output)?;
output.write_chunk(chunk)?;
context.rss.has_newline(chunk.is_newline_terminated());
if context.unbuffered {
output.flush()?;
}
Ok(())
}
/// Write one owned output record.
fn write_buffer_record(
output: &mut OutputBuffer,
context: &mut ProcessingContext,
bytes: &[u8],
has_newline: bool,
) -> UResult<()> {
if bytes.is_empty() {
return Ok(());
}
context.rss.new_record(output)?;
output.write_bytes(bytes)?;
context.rss.has_newline(has_newline);
if context.unbuffered {
output.flush()?;
@@ -386,18 +448,20 @@ fn substitute(
// prints the pre-execution text then executes, while 'ep' executes
// then prints the result.
if sub.print_flag && sub.p_before_e {
write_chunk(output, context, pattern)?;
write_chunk_record(output, context, pattern)?;
}
if sub.execute {
execute_pattern_as_shell_command(pattern, command, context)?;
}
if sub.print_flag && !sub.p_before_e {
write_chunk(output, context, pattern)?;
write_chunk_record(output, context, pattern)?;
}
// Write to file if needed.
if let Some(ref writer) = sub.write_file {
writer.borrow_mut().write_line_bytes(pattern.as_bytes())?;
writer
.borrow_mut()
.write_line_bytes(pattern.as_bytes(), pattern.is_newline_terminated())?;
}
context.substitution_made = true;
}
@@ -465,10 +529,21 @@ fn flush_appends(output: &mut OutputBuffer, context: &mut ProcessingContext) ->
for elem in &context.append_elements {
match elem {
AppendElement::Text(text) => {
context.rss.new_record(output)?;
output.write_bytes(text.as_ref())?;
context.rss.has_newline(text.ends_with(b"\n"));
}
AppendElement::ReaderText(text) => {
context.rss.new_record(output)?;
output.write_bytes(text.as_ref())?;
// GNU sed doesn't care about read file \n ending.
context.rss.has_newline(true);
}
AppendElement::Path(path) => {
context.rss.new_record(output)?;
output.copy_file(path)?;
// GNU sed doesn't care about read file \n ending.
context.rss.has_newline(true);
}
}
}
@@ -608,6 +683,7 @@ fn list(
fn process_address_0(
commands: Option<Rc<RefCell<Command>>>,
output: &mut OutputBuffer,
context: &mut ProcessingContext,
) -> UResult<()> {
// Prescan for zero-address which must produce output
// before any input line is read.
@@ -622,7 +698,10 @@ fn process_address_0(
&& cmd.addr2.is_none()
{
let path = extract_variant!(cmd, Path);
context.rss.new_record(output)?;
output.copy_file(path)?;
// GNU sed doesn't care about read file \n ending.
context.rss.has_newline(true);
}
cmd.next.clone()
@@ -641,7 +720,7 @@ fn process_file(
output: &mut OutputBuffer,
context: &mut ProcessingContext,
) -> UResult<()> {
process_address_0(commands.clone(), output)?;
process_address_0(commands.clone(), output, context)?;
// Loop over the input lines as pattern space.
'lines: while let Some(mut pattern) = reader.get_line()? {
@@ -706,7 +785,7 @@ fn process_file(
pattern.clear();
if command.addr2.is_none() || context.last_address || reader.last_line()? {
let text = extract_variant!(command, Text);
output.write_bytes(text.as_ref())?;
write_buffer_record(output, context, text, true)?;
}
break;
}
@@ -733,10 +812,16 @@ fn process_file(
}
CommandData::Text(cmd_bytes) => {
let shell_out = shell_stdout(cmd_bytes.to_vec(), &command, context)?;
output.write_bytes(&shell_out)?;
write_buffer_record(output, context, &shell_out, true)?;
}
_ => panic!("invalid 'e' command data"),
},
'F' => {
// Output current input file name.
let mut bytes = context.input_name.as_os_str().as_encoded_bytes().to_vec();
bytes.push(b'\n');
write_buffer_record(output, context, &bytes, true)?;
}
'g' => {
// Replace pattern with the contents of the hold space.
pattern.set_to_bytes(context.hold.content.clone(), context.hold.has_newline);
@@ -762,11 +847,13 @@ fn process_file(
'i' => {
// Write text to standard output.
let text = extract_variant!(command, Text);
output.write_bytes(text.as_ref())?;
write_buffer_record(output, context, text, true)?;
}
'l' => {
let width = *extract_variant!(command, Number);
context.rss.new_record(output)?;
list(output, &pattern, width, &command.location, context)?;
context.rss.has_newline(true);
}
'n' => {
break;
@@ -784,25 +871,29 @@ fn process_file(
continue 'lines;
}
'p' => {
write_chunk(output, context, &pattern)?;
write_chunk_record(output, context, &pattern)?;
}
'P' => {
let line = pattern.as_bytes();
if let Some(pos) = memchr(b'\n', line) {
output.write_bytes(&line[..=pos])?;
write_buffer_record(output, context, &line[..=pos], true)?;
} else {
write_chunk(output, context, &pattern)?;
write_chunk_record(output, context, &pattern)?;
}
}
'q' => {
// Quit after printing the pattern space.
set_exit_code(*extract_variant!(command, Number) as i32);
set_exit_code(
i32::try_from(*extract_variant!(command, Number)).unwrap_or(i32::MAX),
);
context.stop_processing = true;
break;
}
'Q' => {
// Quit immediatelly.
set_exit_code(*extract_variant!(command, Number) as i32);
set_exit_code(
i32::try_from(*extract_variant!(command, Number)).unwrap_or(i32::MAX),
);
context.stop_processing = true;
context.quiet = true;
break;
@@ -814,6 +905,15 @@ fn process_file(
.append_elements
.push(AppendElement::Path(path.clone()));
}
'R' => {
// Copy one line from the file to standard output later.
let reader = extract_variant!(command, NamedReader);
if let Some(line) = reader.borrow_mut().read_line() {
context
.append_elements
.push(AppendElement::ReaderText(Rc::from(line)));
}
}
's' => {
substitute(&mut pattern, &command, context, output)?;
}
@@ -831,10 +931,44 @@ fn process_file(
// Branch to the end of the script.
break;
}
'T' if context.substitution_made => {
// A substitution was made since the last cycle or T,
// so reset the flag and fall through without branching.
context.substitution_made = false;
}
'T' => {
// Branch to the specified label or end if none is given
// if no substitution was made since last cycle or T.
let target = extract_variant!(command, BranchTarget);
if target.is_some() {
// New command to execute
current.clone_from(target);
continue;
}
// Branch to the end of the script.
break;
}
'w' => {
// Append the pattern space to the specified file.
let writer = extract_variant!(command, NamedWriter);
writer.borrow_mut().write_line_bytes(pattern.as_bytes())?;
writer
.borrow_mut()
.write_line_bytes(pattern.as_bytes(), pattern.is_newline_terminated())?;
}
'W' => {
// Append only the first line of the pattern space.
let writer = extract_variant!(command, NamedWriter);
let pattern_bytes = pattern.as_bytes();
let (first_line, found_newline) =
match pattern_bytes.iter().position(|&b| b == b'\n') {
// A slice including the newline
Some(pos) => (&pattern_bytes[..=pos], true),
None => (pattern_bytes, false),
};
writer.borrow_mut().write_line_bytes(
first_line,
!found_newline && pattern.is_newline_terminated(),
)?;
}
'x' => {
// Exchange the contents of the pattern and hold spaces.
@@ -861,7 +995,12 @@ fn process_file(
}
'=' => {
// Output current line number.
output.write_str(format!("{}\n", context.line_number))?;
write_buffer_record(
output,
context,
format!("{}\n", context.line_number).as_bytes(),
true,
)?;
}
// The compilation should supply only valid codes.
_ => panic!("invalid command code"),
@@ -871,13 +1010,13 @@ fn process_file(
}
if !context.quiet {
write_chunk(output, context, &pattern)?;
write_chunk_record(output, context, &pattern)?;
}
flush_appends(output, context)?;
if context.stop_processing {
output.flush_pending_newline()?;
context.rss.new_record(output)?;
break;
}
}
@@ -889,10 +1028,7 @@ fn process_file(
{
let mut pending = action.prepend;
pending.push(b'\n');
output.write_bytes(&pending)?;
if context.unbuffered {
output.flush()?;
}
write_buffer_record(output, context, &pending, true)?;
}
Ok(())
@@ -924,11 +1060,11 @@ pub fn process_all_files(
let mut in_place = InPlace::new(context.clone());
let last_file_index = files.len() - 1;
for (index, path) in files.iter().enumerate() {
for (index, path) in files.into_iter().enumerate() {
context.last_file = index == last_file_index;
let mut reader = LineReader::open(path)
let mut reader = LineReader::open(&path)
.map_err_context(|| format!("error opening input file {}", path.quote()))?;
let output = in_place.begin(path)?;
let output = in_place.begin(&path)?;
if context.separate || index == 0 {
context.line_number = 0;
@@ -939,7 +1075,7 @@ pub fn process_all_files(
context.hold.has_newline = true;
}
context.input_name = path.quote().to_string();
context.input_name = path;
process_file(commands.clone(), &mut reader, output, context)?;
// Handle any N command remains.
@@ -950,7 +1086,7 @@ pub fn process_all_files(
{
let mut pending = action.prepend;
pending.push(b'\n');
output.write_bytes(&pending)?;
write_buffer_record(output, context, &pending, true)?;
}
in_place.end()?;
@@ -961,7 +1097,7 @@ pub fn process_all_files(
}
// Flush all output files
named_writer::flush_all()?;
named_io::flush_all()?;
Ok(())
}
@@ -972,6 +1108,65 @@ mod tests {
use std::io::{Read, Seek, SeekFrom};
use tempfile::tempfile;
fn record_separator_output(
state: &mut RecordSeparatorState,
configure: impl FnOnce(&mut RecordSeparatorState),
) -> String {
let mut file = tempfile().unwrap();
let mut output = OutputBuffer::new(Box::new(file.try_clone().unwrap()));
configure(state);
state.new_record(&mut output).unwrap();
output.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut written = String::new();
file.read_to_string(&mut written).unwrap();
written
}
#[test]
fn record_separator_default_does_not_emit_newline() {
let mut state = RecordSeparatorState::default();
assert_eq!(record_separator_output(&mut state, |_| {}), "");
}
#[test]
fn record_separator_terminates_unterminated_record() {
let mut state = RecordSeparatorState::default();
assert_eq!(
record_separator_output(&mut state, |state| state.has_newline(false)),
"\n"
);
}
#[test]
fn record_separator_emits_missing_newline_only_once() {
let mut state = RecordSeparatorState::default();
state.has_newline(false);
let mut file = tempfile().unwrap();
let mut output = OutputBuffer::new(Box::new(file.try_clone().unwrap()));
state.new_record(&mut output).unwrap();
state.new_record(&mut output).unwrap();
output.flush().unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
let mut written = String::new();
file.read_to_string(&mut written).unwrap();
assert_eq!(written, "\n");
}
#[test]
fn record_separator_newline_terminated_record_needs_no_separator() {
let mut state = RecordSeparatorState::default();
assert_eq!(
record_separator_output(&mut state, |state| state.has_newline(true)),
""
);
}
#[test]
fn test_readable_ascii_byte_named_escapes() {
assert_eq!(readable_ascii_byte(b'\n'), r"\012");
+291 -3
View File
@@ -38,7 +38,18 @@ fn test_version() {
.arg("--version")
.succeeds()
.no_stderr()
.stdout_is(short);
.stdout_is(&short);
assert!(short.contains("uutils"));
}
#[test]
fn test_help_mentions_uutils() {
new_ucmd!()
.arg("--help")
.succeeds()
.no_stderr()
.stdout_contains("part of uutils");
}
#[test]
@@ -1077,6 +1088,15 @@ fn test_e_command_with_arg_non_utf8_output_passthrough() {
.stdout_is_bytes(b"\xff\na\n");
}
#[cfg(unix)]
#[test]
fn test_e_command_no_newline_ignored() {
new_ucmd!()
.args(&["3e printf hi", LINES1])
.succeeds()
.stdout_is_fixture_bytes("output/e_command_no_newline_ignored");
}
////////////////////////////////////////////////////////////
// Transliteration: y
check_output!(trans_simple, ["-e", r"y/0123456789/9876543210/", LINES1]);
@@ -1414,6 +1434,69 @@ tb"#,
]
);
// T branches when NO substitution was made (inverse of t).
// Lines not ending in "2" leave s/2$/X/ with no match, so T branches to
// :skip and keeps the original line; lines ending in "2" fall through and
// get the "SUB " mark.
check_output!(
branch_no_sub_simple,
[
"-n",
"-e",
r#"
s/2$/X/
Tskip
s/^/SUB /
:skip
p
"#,
LINES1
]
);
// Check that T also clears the substitution-done flag, matching t.
// After s/// sets the flag, the first T sees a substitution so it does
// not branch but still clears the flag; the second T then sees no
// substitution and branches to :y. Reaching :y (CLEARED) rather than
// falling through to :x (NOTCLEARED) proves the first T cleared the flag.
check_output!(
branch_no_sub_clears,
[
"-n",
"-e",
r#"
s/^/^/
Tx
Ty
:x
s/^/NOTCLEARED /p
b
:y
s/^/CLEARED /p
"#,
LINES1
]
);
#[test]
fn test_branch_no_sub_non_posix() {
new_ucmd!()
.args(&["--posix", "T"])
.fails()
.code_is(1)
.stderr_contains("invalid command code");
}
check_output!(
multiple_no_newline,
[
"-n",
"p",
"input/no-new-line.txt",
"input/no-new-line.txt",
"input/no-new-line.txt",
]
);
////////////////////////////////////////////////////////////
// Text: a, c, i
@@ -1582,8 +1665,32 @@ check_output!(
);
////////////////////////////////////////////////////////////
// r, w commands
// r, R, w, W commands
check_output!(read_ok, [format!("4r {LINES2}"), LINES1.to_string()]);
check_output!(read_no_newline, ["4r input/no-new-line.txt", LINES1]);
// r Doesn't doesn't record lacking newline
check_output!(
read_no_newline_update,
[
"-e",
"4r input/no-new-line.txt",
"-e",
"4r input/no-new-line.txt",
LINES1
]
);
// r respects a lacking newline and adds it and also resets the state
// to no newline needed.
check_output!(
read_no_newline_respect,
[
"-e",
"s/^/i: /;r input/no-new-line.txt",
"-e",
"r input/no-new-line.txt",
"input/no-new-line.txt"
]
);
check_output!(read_missing, ["5r /xyzzyxyzy42", LINES1]);
check_output!(read_empty, ["6r input/empty", LINES1]);
check_output!(
@@ -1603,6 +1710,25 @@ fn sandbox_rejects_read_command() {
.stderr_contains("command not allowed with --sandbox");
}
check_output!(read_one, ["6R input/lines2", LINES1]);
check_output!(
read_one_twice,
["-e", "5R input/lines2", "-e", "7R input/lines2", LINES1]
);
check_output!(read_one_many, ["R input/lines2", LINES1]);
check_output!(read_one_empty, ["R input/empty", LINES1]);
check_output!(read_one_missing, ["R input/xyzzy42", LINES1]);
check_output!(read_one_no_newline, ["R input/no-new-line.txt", LINES1]);
#[test]
fn read_one_line_rejected_in_posix_mode() {
new_ucmd!()
.args(&["--posix", "R /tmp/read-one-line"])
.fails()
.code_is(1)
.stderr_is("sed: <script argument 1>:1:1: error: invalid command code `R'\n");
}
#[test]
fn sandbox_rejects_subst_write_flag() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
@@ -1637,6 +1763,23 @@ fn sandbox_rejects_write_command() -> std::io::Result<()> {
Ok(())
}
#[test]
fn sandbox_rejects_first_line_write_command() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
let cmd = format!("W {}", temp.path().display());
new_ucmd!()
.args(&["--sandbox", &cmd, LINES1])
.fails()
.stderr_contains("command not allowed with --sandbox");
let mut actual = String::new();
temp.reopen()?.read_to_string(&mut actual)?;
assert!(actual.is_empty());
Ok(())
}
#[test]
fn write_single_file() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
@@ -1653,6 +1796,44 @@ fn write_single_file() -> std::io::Result<()> {
Ok(())
}
#[test]
fn write_single_file_twice() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
// Write odd and even lines to the same file through different commands.
// Unless the writers are unified, this fails on Windows.
let cmd1 = format!("3~2w {}", temp.path().display());
let cmd2 = format!("4~2w {}", temp.path().display());
new_ucmd!()
.args(&["-e", &cmd1, "-e", &cmd2, "-e", "12q", LINES1])
.succeeds();
let mut actual = String::new();
temp.reopen()?.read_to_string(&mut actual)?;
let expected = fs::read_to_string("tests/fixtures/sed/output/write_single_file")?;
assert_eq!(actual, expected, "Output did not match fixture");
Ok(())
}
#[test]
fn write_single_file_no_newline() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
let cmd = format!("w {}", temp.path().display());
new_ucmd!()
.args(&[cmd.as_str(), "input/no-new-line.txt"])
.succeeds();
let mut actual = String::new();
temp.reopen()?.read_to_string(&mut actual)?;
assert_eq!(actual, "Hello", "Output did not match expected");
Ok(())
}
#[test]
fn write_two_files() -> std::io::Result<()> {
let temp1 = NamedTempFile::new()?;
@@ -1680,8 +1861,51 @@ fn write_two_files() -> std::io::Result<()> {
Ok(())
}
#[test]
fn write_first_line_newline() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
let cmd = format!("N;W {}", temp.path().display());
new_ucmd!()
.args(&["-n", "-e", &cmd])
.pipe_in("abc\ndef\n")
.succeeds();
let mut actual = String::new();
temp.reopen()?.read_to_string(&mut actual)?;
assert_eq!(actual, "abc\n");
Ok(())
}
#[test]
fn write_first_line_no_newline() -> std::io::Result<()> {
let temp = NamedTempFile::new()?;
let cmd = format!("W {}", temp.path().display());
new_ucmd!()
.args(&["-n", "-e", &cmd])
.pipe_in("abc")
.succeeds();
let mut actual = String::new();
temp.reopen()?.read_to_string(&mut actual)?;
assert_eq!(actual, "abc");
Ok(())
}
#[test]
fn write_first_line_with_w_command_is_non_posix() {
new_ucmd!()
.args(&["--posix", "W /tmp/out"])
.fails()
.code_is(1)
.stderr_is("sed: <script argument 1>:1:1: error: invalid command code `W'\n");
}
////////////////////////////////////////////////////////////
// =, l commands
// =, l, F commands
check_output!(number_continuous, ["/l2_/=", LINES1, LINES2]);
check_output!(number_separate, ["-s", "/l._8/=", LINES1, LINES2]);
check_output!(number_range, ["-e", "10,12=", LINES1]);
@@ -1690,6 +1914,28 @@ check_output!(number_range_out_of_bounds, ["-e", "47,60=", LINES1]);
check_output!(list_ascii, ["-n", "l 60", "input/ascii"]);
check_output!(list_empty, ["-n", "l 60", "input/empty"]);
check_output!(filename_file, ["-n", r"F", LINES1]);
// Non-ASCII filename
check_output!(filename_αρχείο1, [r"F", "input/αρχείο1"]);
#[test]
fn filename_stdin() {
new_ucmd!()
.args(&["-n", "F"])
.pipe_in("a\nb\n")
.succeeds()
.stdout_is("-\n-\n");
}
#[test]
fn filename_non_posix() {
new_ucmd!()
.args(&["--posix", "F"])
.fails()
.code_is(1)
.stderr_contains("invalid command code");
}
/// List Unicode input under an explicit UTF-8 locale.
#[test]
fn list_unicode() {
@@ -2208,6 +2454,48 @@ fn test_print_first_line_no_newline() {
.stdout_is("foo");
}
#[test]
fn test_expected_newer_version() {
new_ucmd!()
.args(&["v4.10"])
.fails()
.stderr_is("sed: <script argument 1>:1:6: error: expected newer version of sed\n");
}
#[test]
fn test_invalid_version() {
new_ucmd!()
.args(&["v4.a"])
.fails()
.stderr_is("sed: <script argument 1>:1:5: error: invalid version of sed\n");
}
#[test]
fn test_valid_version() {
new_ucmd!().args(&["v4.9"]).succeeds();
}
#[test]
fn test_valid_only_major_version() {
// v4.0.0
new_ucmd!().args(&["v4"]).succeeds();
}
#[test]
fn test_invalid_only_major_version() {
// v999.0.0
new_ucmd!()
.args(&["v999"])
.fails()
.stderr_is("sed: <script argument 1>:1:5: error: invalid version of sed\n");
}
#[test]
fn test_default_version() {
// defaults to v4.9.0
new_ucmd!().args(&["v"]).succeeds();
}
//--posix should reject GNU substitute flags i/I https://github.com/uutils/sed/issues/401
#[test]
fn test_posix_reject_flags() {
+14
View File
@@ -0,0 +1,14 @@
l1_1
l1_2
l1_3
l1_4
l1_5
l1_6
l1_7
l1_8
l1_9
l1_10
l1_11
l1_12
l1_13
l1_14
+14
View File
@@ -0,0 +1,14 @@
CLEARED ^l1_1
CLEARED ^l1_2
CLEARED ^l1_3
CLEARED ^l1_4
CLEARED ^l1_5
CLEARED ^l1_6
CLEARED ^l1_7
CLEARED ^l1_8
CLEARED ^l1_9
CLEARED ^l1_10
CLEARED ^l1_11
CLEARED ^l1_12
CLEARED ^l1_13
CLEARED ^l1_14
+14
View File
@@ -0,0 +1,14 @@
l1_1
SUB l1_X
l1_3
l1_4
l1_5
l1_6
l1_7
l1_8
l1_9
l1_10
l1_11
SUB l1_1X
l1_13
l1_14
+14
View File
@@ -0,0 +1,14 @@
l1_1
l1_2
hil1_3
l1_4
l1_5
l1_6
l1_7
l1_8
l1_9
l1_10
l1_11
l1_12
l1_13
l1_14
+14
View File
@@ -0,0 +1,14 @@
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
input/lines1
Loaded 20 of 32 files, more files were not shown because too many files have changed in this diff. Show more