mattsu
3edf14eefd
* chmod:fix safe traversal/access (#9554) * feat(chmod): use dirfd for recursive subdirectory traversal - Update chmod recursive logic to use directory file descriptors instead of full paths for subdirectories - Improves performance, avoids path length issues, and ensures dirfd-relative openat calls - Add test to verify strace output shows no AT_FDCWD with multi-component paths * test(chmod): add spell-check ignore for dirfd, subdirs, openat, FDCWD Added a spell-checker ignore directive in the chmod test file to suppress false positives for legitimate technical terms used in Unix API calls. * test(chmod): enforce strace requirement in recursive test, fail fast instead of skip Previously, the test_chmod_recursive_uses_dirfd_for_subdirs test skipped gracefully if strace was unavailable, without failing. This change enforces the strace dependency by failing the test immediately if strace is not installed or runnable, ensuring the test runs reliably in environments where it is expected to pass, and preventing silent skips. * ci: install strace in Ubuntu CI jobs for debugging system calls Add installation of strace tool on Ubuntu runners in both individual build/test and feature build/test jobs. This enables tracing system calls during execution, aiding in debugging and performance analysis within the CI/CD pipeline. Updated existing apt-get commands and added conditional steps for Linux-only installations. * ci: Add strace installation to Ubuntu-based CI workflows Install strace on ubuntu-latest runners across multiple jobs to enable system call tracing for testing purposes, ensuring compatibility with tests that require this debugging tool. This includes updating package lists in existing installation steps. * chore(build): install strace and prevent apt prompts in Cross.toml pre-build Modified the pre-build command to install strace utility for debugging and added -y flag to apt-get install to skip prompts, ensuring non-interactive builds. * feat(build): support Alpine-based cross images in pre-build Detect package manager (apt vs apk) to install tzdata and strace in both Debian/Ubuntu and Alpine *-musl targets. Added fallback warning for unsupported managers. This ensures strace is available for targets using Alpine, which doesn't have apt-get. * refactor(build): improve pre-build script readability by using multi-line strings Replace escaped multi-line string with triple-quoted string for better readability in Cross.toml. * feat(ci): install strace in WSL2 GitHub Actions workflow Install strace utility in the WSL2 environment to support tracing system calls during testing. Minor update to Cross.toml spell-checker ignore list for consistency with change. * ci(wsl2): install strace as root with non-interactive apt-get Updated the WSL2 workflow step to use root shell (wsl-bash-root) for installing strace, removing sudo calls and adding DEBIAN_FRONTEND=noninteractive to prevent prompts. This improves CI reliability by ensuring direct root access and automated, interrupt-free package installation. * ci: Move strace installation to user shell and update spell ignore Fix WSL2 GitHub Actions workflow by installing strace as the user instead of root for better permission handling, and add "noninteractive" to the spell-checker ignore comment for consistency with the new apt-get command. This ensures the tool is available in the testing environment without unnecessary privilege escalation. * chore: ci: remove unused strace installation from CI workflows Remove strace package installation from multiple GitHub Actions workflow files (CICD.yml, l10n.yml, wsl2.yml). Strace was historically installed in Ubuntu jobs for debugging system calls, but it's no longer required for the tests and builds, reducing CI setup time and dependencies. * ci: add strace installation and fix spell-checker comments in CI files - Install strace package in CICD workflow to support safe traversal verification for utilities like rm, chmod, chown, chgrp, mv, and du, enabling syscall tracing for testing. - Clean up spell-checker ignore comments in wsl2.yml and Cross.toml by removing misplaced flags.第二个测试产品**ci: add strace installation and fix spell-checker comments in CI files** - Install strace package in CICD workflow to support safe traversal verification for utilities like rm, chmod, chown, chgrp, mv, and du, enabling syscall tracing for testing. - Clean up spell-checker ignore comments in wsl2.yml and Cross.toml by removing misplaced flags. * test: add regression guard for recursive chmod dirfd-relative traversal Add a check in check-safe-traversal.sh to ensure recursive chmod operations use dirfd-relative openat calls instead of AT_FDCWD with multi-component paths, preventing potential race conditions. Ignore the corresponding Rust test as it is now covered by this shell script guard. * Merge pull request #9561 from ChrisDryden/seq_benches seq: adding large integers benchmarks * install: do not call chown when called as root - `pseudo` is a tool which simulates being root by intercepting calls to e.g. `geteuid` and `chown` (by using the `LD_PRELOAD` mechanism). This is used e.g. to build filesystems for embedded devices without running as root on the build machine. - the `chown` call getting removed in this commit does not work when running with `pseudo` and using `PSEUDO_IGNORE_PATHS`: in this case, the call to `geteuid()` gets intercepted by `libpseudo.so` and returns 0, however the call to `chown()` isn't intercepted by `libpseudo.so` in case it is in a path from `PSEUDO_IGNORE_PATHS`, and will thus fail since the process is not really root - the call to `chown()` was added in https://github.com/uutils/coreutils/pull/5735 with the intent of making the test `install-C-root.sh` pass, however it isn't required (GNU coreutils also does not call `chown` just because `install` was called as root) Fixes https://github.com/uutils/coreutils/issues/9116 Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> * du: handle `--files0-from=-` with piped in `-` (#8985) * du: handle --files0-from=- with piped in '-' * build-gnu.sh: remove incorrect string replacement in tests/du/files0-from.pl --------- Co-authored-by: Sylvestre Ledru <sylvestre@debian.org> * perf: optimize rm prompts by reusing stat data to avoid extra syscalls This change adds inline functions for checking file modes and refactors prompt functions to accept pre-fetched stat data. It modifies safe_remove_* functions to handle paths without parents and updates safe_remove_dir_recursive to fetch and reuse initial mode. This reduces redundant statx system calls, improving performance during recursive removals. * feat(rm/linux): Refine interactive file removal prompts - Add specific prompts for symlinks and empty files in 'Always' mode - Refactor matching logic for better clarity and to match GNU rm behavior - Improve handling of write-protected and non-terminal stdin scenarios This enhances the user experience by providing more accurate and targeted confirmations during file removal on Linux. * refactor(rm/linux): reformat prompt_yes! macros for improved readability Refactored multiple call sites of the prompt_yes! macro in linux.rs to use consistent multi-line formatting, enhancing code readability and adhering to style guidelines without altering functionality. Adjusted import ordering slightly for better organization. * refactor(src/uu/rm/src/platform/linux.rs): remove unused 'self' import from std::io Removed the unused 'self' import from the std::io module to clean up the code and avoid potential confusion, as it was not referenced anywhere in the file. This is a minor refactoring for better maintainability. * chore(spell-checker): update ignore list to include statx and behaviour Add "statx" (a Linux system call name) and "behaviour" (potential spelling variant) to the spell-checker ignore comment in the rm utility's Linux platform code, preventing false positives in linting. * fix(linux/rm): correct prompting logic for write-protected files in Interactive::Always mode Refactor the prompt_file_with_stat function in src/uu/rm/src/platform/linux.rs to fix inconsistent prompting for Interactive::Always. Previously, it always used non-protected wording regardless of file writability. Now, it checks if the file is writable and uses appropriate messaging (simple for writable, protected for non-writable). The match logic for Interactive::Once and PromptProtected is also simplified using a triple condition for better readability and to ensure empty vs non-empty protected files are distinguished correctly, matching expected rm behavior. * style(rm): wrap long line in prompt_file_with_stat macro for readability Reformatted the prompt_yes! macro call across multiple lines to improve code readability and adhere to line length conventions. No functional changes. --------- Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Co-authored-by: Chris Dryden <christopher.paul.dryden@gmail.com> Co-authored-by: Etienne Cordonnier <ecordonnier@snap.com> Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com> Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
uutils coreutils
uutils coreutils is a cross-platform reimplementation of the GNU coreutils in Rust. While all programs have been implemented, some options might be missing or different behavior might be experienced.
We provide prebuilt binaries at https://github.com/uutils/coreutils/releases/latest . It is recommended to install from main branch if you install from source.
To install it:
cargo install --git https://github.com/uutils/coreutils coreutils
# cargo install --git https://github.com/uutils/coreutils uu_true # for one util only
~/.cargo/bin/coreutils
Goals
uutils coreutils aims to be a drop-in replacement for the GNU utils. Differences with GNU are treated as bugs.
Our key objectives include:
- Matching GNU's output (stdout and error code) exactly
- Better error messages
- Providing comprehensive internationalization support (UTF-8)
- Improved performances
- Extensions when relevant (example: --progress)
uutils aims to work on as many platforms as possible, to be able to use the same utils on Linux, macOS, Windows and other platforms. This ensures, for example, that scripts can be easily transferred between platforms.
Documentation
uutils has both user and developer documentation available:
Both can also be generated locally, the instructions for that can be found in the coreutils docs repository.
Use weblate/rust-coreutils to translate the Rust coreutils into your language.
Requirements
- Rust (
cargo,rustc) - GNU Make (optional)
Rust Version
uutils follows Rust's release channels and is tested against stable, beta and
nightly. The current Minimum Supported Rust Version (MSRV) is 1.85.0.
Building
There are currently two methods to build the uutils binaries: either Cargo or GNU Make.
Building the full package, including all documentation, requires both Cargo and GNU Make on a Unix platform.
For either method, we first need to fetch the repository:
git clone https://github.com/uutils/coreutils
cd coreutils
Cargo
Building uutils using Cargo is easy because the process is the same as for every other Rust program:
cargo build --release
Replace --release with --profile=release-fast or --profile=release-small to use all optimizations or save binary size.
This command builds the most portable common core set of uutils into a multicall (BusyBox-type) binary, named 'coreutils', on most Rust-supported platforms.
Additional platform-specific uutils are often available. Building these expanded sets of uutils for a platform (on that platform) is as simple as specifying it as a feature:
cargo build --release --features macos
# or ...
cargo build --release --features windows
# or ...
cargo build --release --features unix
To build SELinux-specific features, including chcon and runcon, ensure that libselinux
and libclang are installed on your system. Then, run the following command:
cargo build --release --features unix,feat_selinux
If you don't want to build every utility available on your platform into the final binary, you can also specify which ones you want to build manually. For example:
cargo build --features "base32 cat echo rm" --no-default-features
If you want to build the utilities as individual binaries, that is also possible:
cargo build --release --bins --workspace --exclude coreutils --exclude uu_runcon --exclude uu_chcon
Each utility is contained in its own package within the main repository, named "uu_UTILNAME". To
build selected individual utilities, use the --package [aka -p] option. For example:
cargo build -p uu_base32 -p uu_cat -p uu_echo -p uu_rm
GNU Make
Building using make is a simple process as well.
To simply build all available utilities (with debug profile):
make
In release-fast mode:
make PROFILE=release-fast
To build all but a few of the available utilities:
make SKIP_UTILS='UTILITY_1 UTILITY_2'
To build only a few of the available utilities:
make UTILS='UTILITY_1 UTILITY_2'
Installation
Install with Cargo
Likewise, installing can simply be done using:
cargo install --path . --locked
This command will install uutils into Cargo's bin folder (e.g.
$HOME/.cargo/bin).
This does not install files necessary for shell completion or manpages. For
manpages or shell completion to work, use GNU Make or see
Manually install shell completions/Manually install manpages.
Install with GNU Make
To install all available utilities:
make install
To install all utilities with all possible optimizations:
make PROFILE=release-fast install
To install using sudo switch -E must be used:
sudo -E make install
To install all but a few of the available utilities:
make SKIP_UTILS='UTILITY_1 UTILITY_2' install
To install only a few of the available utilities:
make UTILS='UTILITY_1 UTILITY_2' install
To install every program with a prefix (e.g. uu-echo uu-cat):
make PROG_PREFIX=uu- install
PROG_PREFIX requires separator -, _, or =.
To install the multicall binary:
make MULTICALL=y install
Set install parent directory (default value is /usr/local):
# DESTDIR is also supported
make PREFIX=/my/path install
Installing with make installs shell completions for all installed utilities
for bash, fish and zsh. Completions for elvish and powershell can also
be generated; See Manually install shell completions.
To skip installation of completions and manpages:
make COMPLETIONS=n MANPAGES=n install
Manually install shell completions
The uudoc binary generates completions for the bash, elvish,
fish, powershell and zsh shells to stdout.
Install uudoc by
cargo install --bin uudoc --features uudoc --path .
Then use the installed binary:
uudoc completion <utility> <shell>
So, to install completions for ls on bash to
/usr/local/share/bash-completion/completions/ls, run:
uudoc completion ls bash > /usr/local/share/bash-completion/completions/ls.bash
Completion for prefixed cp with uu- on zsh is generated by
env PROG_PREFIX=uu- uudoc completion cp zsh
Manually install manpages
To generate manpages, the syntax is:
uudoc manpage <utility>
So, to install the manpage for ls to /usr/local/share/man/man1/ls.1 run:
uudoc manpage ls > /usr/local/share/man/man1/ls.1
Un-installation
Un-installation differs depending on how you have installed uutils. If you used Cargo to install, use Cargo to uninstall. If you used GNU Make to install, use Make to uninstall.
Uninstall with Cargo
To uninstall uutils:
cargo uninstall coreutils
Uninstall with GNU Make
To uninstall all utilities:
make uninstall
To uninstall every program with a set prefix:
make PROG_PREFIX=uu- uninstall
To uninstall the multicall binary:
make MULTICALL=y uninstall
To uninstall from a custom parent directory:
# DESTDIR is also supported
make PREFIX=/my/path uninstall
GNU test suite compatibility
Below is the evolution of how many GNU tests uutils passes. A more detailed breakdown of the GNU test results of the main branch can be found in the user manual.
See https://github.com/orgs/uutils/projects/1 for the main meta bugs (many are missing).
Contributing
To contribute to uutils, please see CONTRIBUTING.
License
uutils is licensed under the MIT License - see the LICENSE file for details
GNU Coreutils is licensed under the GPL 3.0 or later.