Follow-up to the metadata TOCTOU fix (#11972, PR #12439):
- Add a syscall-level regression guard in util/check-safe-traversal.sh:
under strace, head must read metadata from the open descriptor
(fstat/statx on the fd) and must not stat the path before opening it.
Verified this fails on the pre-fix code and passes after.
- Add an integration test asserting that an unreadable file produces an
error but no "==> name <==" header, matching GNU (the header is only
printed after a successful open).
- Fix a stray paren in the Windows-branch comment.
`pr -e0` on input containing a tab aborted with a remainder-by-zero panic
(`chunk.len() % expand_options.width` in `apply_expand_tab`). The
`-e[char][width]` parser accepted a width of `0` — and a negative width via
e.g. `-eX-5` — without validation, so the zero/negative reached the chunk
arithmetic.
Reject `width <= 0` while parsing `-e`, emitting the same invalid-argument
error GNU produces (`pr: '-e' extra characters or invalid number in the
argument: '0'`, exit 1) instead of crashing.
* shuf: fix panic/abort on large -i range without small --head-count
NonrepeatingIterator::new sized its sparse hash map to
min(head_count, range_len) and allocated it with the infallible
with_capacity_and_hasher. head_count defaults to u64::MAX when -n is
absent (and can be passed a huge value explicitly), so for a large -i
range the map was asked to reserve the whole range:
shuf -i 1-9999999999999999999 # no -n
shuf -i 1-9999999999999999999 -n 9999999999999999999
Both aborted (exit 134) with a hashbrown "Hash table capacity overflow"
panic or an allocator abort, where GNU prints "memory exhausted" and
exits 1.
Reserve fallibly with try_reserve (mirroring the Vec branch) and map
the failure to a clean error, so an unsatisfiable request errors like
GNU instead of crashing. A small --head-count still works unchanged.
Closes#12500.
* shuf: assert full stderr (incl. program prefix) in memory-exhausted tests
Per review on #12501: switch the two memory-exhausted regression tests
from stderr_contains to stderr_only("shuf: memory exhausted\n") so they
verify the whole stderr, including the `shuf:` program-name prefix.
sort -m takes: 3 lines, 96003 bytes
and emits: 4 lines, 96004 bytes
The output line lengths before the fix are:
```
a x 32000
b x 23809
b x 8191
c x 32000
```
So it splits one of the lines into two (23809 + 8191 = 32000).
In addition, the output becomes unsorted because the shorter 'b' fragment sorts before the longer 'b' fragment.
The issue is that in `chunks.rs`, `sep_iter` is relatve to `search_start`. But the returned value needs to be absolute position relative to the `buffer`.
We end up with these particular numbers because
- in merge.rs, initial chunk is created as `RecycledChunk::new(8 * 1024)` (8192 bytes)
- `search_start = 8192`; newline is at absolute buffer index `32000`
- `memchr_iter` returns `32000 - 8192 = 23808`, and newline adds + 1 byte
Fix doc comments in safe_traversal that still described the old behavior
of replacing symlinks with real directories. Rename and deduplicate tests
that were originally written as race-condition regression tests but now
just verify symlink-following behavior.
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
## Summary
Fixes#11469
`install -D` was replacing pre-existing symlinks in the destination path with real directories instead of following them. This broke any workflow where part of the install prefix is a symlink; including BOSH deployments, Homebrew, Nix, stow, and any `make install` targeting a symlinked prefix.
**Reproduction (from the issue):**
```sh
mkdir -p /tmp/target
ln -s /tmp/target /tmp/link
echo hello > /tmp/file.txt
install -D -m 644 /tmp/file.txt /tmp/link/subdir/file.txt
# GNU coreutils 8.32: /tmp/link stays a symlink, file lands in /tmp/target/subdir/file.txt
# uutils 0.7.0: /tmp/link is replaced with a real directory — wrong
```
## Root cause
PR #10140 introduced `create_dir_all_safe()` in `safe_traversal.rs` to prevent TOCTOU symlink race conditions. The fix was correct in intent but too aggressive: `open_or_create_subdir()` unconditionally unlinked and recreated any symlink it encountered, including pre-existing legitimate ones.
## Changes
**`src/uucore/src/lib/features/safe_traversal.rs`**
- `open_or_create_subdir`: when `stat_at` returns `S_IFLNK`, call `open_subdir(Follow)` instead of `unlink_at + mkdir_at`. The `O_DIRECTORY` flag already in `open_subdir` means dangling or non-directory symlinks still return an error cleanly.
- `find_existing_ancestor`: switch from `fs::symlink_metadata` to `fs::metadata` so that a symlink-to-directory is recognised as an existing ancestor rather than a component to recreate (this was already the stated intent in the function's doc comment).
**`src/uu/install/src/install.rs`**
- Align the `dir_exists` check and the `DirFd::open` call to also follow symlinks, consistent with the above.
**`tests/by-util/test_install.rs`**
- Update the two tests added by #10140 — they were asserting the buggy behavior (symlink replaced). Flip the assertions to document the correct GNU behavior.
- Add `test_install_d_follows_symlink_prefix` as a direct regression test for the issue's reproduction case.
## TOCTOU / security note
The true TOCTOU race (a symlink *injected during the operation* into a not-yet-existing path component) is still blocked: `mkdirat` fails with `EEXIST` if an attacker creates a symlink between `stat_at` returning `ENOENT` and our `mkdir_at`. Newly-created directories are still opened with `O_NOFOLLOW`.
What changes is that *pre-existing* symlinks are now followed — which is exactly what GNU coreutils 8.32 does. The previous behavior was stricter than GNU in this regard.
