mirror of
https://github.com/uutils/grep.git
synced 2026-06-10 16:15:11 -07:00
Initial import
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
[target.x86_64-unknown-redox]
|
||||
linker = "x86_64-unknown-redox-gcc"
|
||||
[target.aarch64-unknown-linux-gnu]
|
||||
linker = "aarch64-linux-gnu-gcc"
|
||||
[target.riscv64gc-unknown-linux-musl]
|
||||
rustflags = ["-C", "target-feature=+crt-static"]
|
||||
[target.'cfg(target_env = "msvc")']
|
||||
rustflags = ["-C", "target-feature=+crt-static"]
|
||||
@@ -0,0 +1 @@
|
||||
/target
|
||||
@@ -0,0 +1,128 @@
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our
|
||||
community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socioeconomic status,
|
||||
nationality, personal appearance, race, religion, or sexual identity
|
||||
and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the
|
||||
overall community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or
|
||||
advances of any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email
|
||||
address, without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Enforcement Responsibilities
|
||||
|
||||
Community leaders are responsible for clarifying and enforcing our standards of
|
||||
acceptable behavior and will take appropriate and fair corrective action in
|
||||
response to any behavior that they deem inappropriate, threatening, offensive,
|
||||
or harmful.
|
||||
|
||||
Community leaders have the right and responsibility to remove, edit, or reject
|
||||
comments, commits, code, wiki edits, issues, and other contributions that are
|
||||
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
||||
decisions when appropriate.
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces, and also applies when
|
||||
an individual is officially representing the community in public spaces.
|
||||
Examples of representing our community include using an official e-mail address,
|
||||
posting via an official social media account, or acting as an appointed
|
||||
representative at an online or offline event.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the community leaders responsible for enforcement at
|
||||
sylvestre@debian.org.
|
||||
All complaints will be reviewed and investigated promptly and fairly.
|
||||
|
||||
All community leaders are obligated to respect the privacy and security of the
|
||||
reporter of any incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series
|
||||
of actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or
|
||||
permanent ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within
|
||||
the community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.0, available at
|
||||
<https://www.contributor-covenant.org/version/2/0/code_of_conduct.html>.
|
||||
|
||||
Community Impact Guidelines were inspired by [Mozilla's code of conduct
|
||||
enforcement ladder](https://github.com/mozilla/diversity).
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
<https://www.contributor-covenant.org/faq>. Translations are available at
|
||||
<https://www.contributor-covenant.org/translations>.
|
||||
+260
@@ -0,0 +1,260 @@
|
||||
<!-- spell-checker:ignore reimplementing toybox RUNTEST CARGOFLAGS nextest embeddable Rustonomicon rustdoc's -->
|
||||
|
||||
# Contributing to grep
|
||||
|
||||
Hi! Welcome to uutils/grep!
|
||||
|
||||
Thanks for wanting to contribute to this project! This document explains
|
||||
everything you need to know to contribute. Before you start make sure to also
|
||||
check out these documents:
|
||||
|
||||
- Our community's [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md).
|
||||
- [DEVELOPMENT.md](./DEVELOPMENT.md) for setting up your development
|
||||
environment.
|
||||
|
||||
Now follows a very important warning:
|
||||
|
||||
> [!WARNING]
|
||||
> uutils is original code and cannot contain any code from GNU or
|
||||
> other implementations. This means that **we cannot accept any changes based on
|
||||
> the GNU source code**. To make sure that cannot happen, **you cannot link to
|
||||
> the GNU source code** either. It is however possible to look at other implementations
|
||||
> under a BSD or MIT license like [Apple's implementation](https://opensource.apple.com/source/file_cmds/)
|
||||
> or [OpenBSD](https://github.com/openbsd/src/tree/master/bin).
|
||||
|
||||
Finally, feel free to join our [Discord](https://discord.gg/wQVJbvJ)!
|
||||
|
||||
<!-- TODO: Getting Oriented -->
|
||||
|
||||
## Design Goals
|
||||
|
||||
We have the following goals with our development:
|
||||
|
||||
- **Compatible**: The utilities should be a drop-in replacement for the GNU
|
||||
coreutils.
|
||||
- **Cross-platform**: All utilities should run on as many of the supported
|
||||
platforms as possible.
|
||||
- **Reliable**: The utilities should never unexpectedly fail.
|
||||
- **Performant**: Our utilities should be written in fast idiomatic Rust. We aim
|
||||
to match or exceed the performance of the GNU utilities.
|
||||
- **Well-tested**: We should have a lot of tests to be able to guarantee
|
||||
reliability and compatibility.
|
||||
|
||||
## How to Help
|
||||
|
||||
There are several ways to help and writing code is just one of them. Reporting
|
||||
issues and writing documentation are just as important as writing code.
|
||||
|
||||
### Reporting Issues
|
||||
|
||||
We can't fix bugs we don't know about, so good issues are super helpful! Here
|
||||
are some tips for writing good issues:
|
||||
|
||||
- If you find a bug, make sure it's still a problem on the `main` branch.
|
||||
- Search through the existing issues to see whether it has already been
|
||||
reported.
|
||||
- Make sure to include all relevant information, such as:
|
||||
- Which version of uutils did you check?
|
||||
- Which version of GNU coreutils are you comparing with?
|
||||
- What platform are you on?
|
||||
- Provide a way to reliably reproduce the issue.
|
||||
- Be as specific as possible!
|
||||
|
||||
### Writing Documentation
|
||||
|
||||
There's never enough documentation. If you come across any documentation that
|
||||
could be improved, feel free to submit a PR for it!
|
||||
|
||||
### Writing Code
|
||||
|
||||
If you want to submit a PR, make sure that you've discussed the solution with
|
||||
the maintainers beforehand. We want to avoid situations where you put a lot of
|
||||
work into a fix that we can't merge! If there's no issue for what you're trying
|
||||
to fix yet, make one _before_ you start working on the PR.
|
||||
|
||||
Generally, we try to follow what GNU is doing in terms of options and behavior.
|
||||
It is recommended to look at the GNU Grep manual
|
||||
([on the web](https://www.gnu.org/software/grep/manual/html_node/index.html),
|
||||
or locally using `info <utility>`). It is more in depth than the man pages and
|
||||
provides a good description of available features and their implementation
|
||||
details. But remember, you cannot look at the GNU source code!
|
||||
|
||||
Also remember that we can only merge PRs which pass our test suite, follow
|
||||
rustfmt, and do not have any warnings from clippy. See
|
||||
[DEVELOPMENT.md](./DEVELOPMENT.md) for more information. Be sure to also read
|
||||
about our [Rust style](#our-rust-style).
|
||||
|
||||
## Our Rust Style
|
||||
|
||||
We want uutils to be written in idiomatic Rust, so here are some guidelines to
|
||||
follow. Some of these are aspirational, meaning that we don't do them correctly
|
||||
everywhere in the code. If you find violations of the advice below, feel free to
|
||||
submit a patch!
|
||||
|
||||
### Don't `panic!`
|
||||
|
||||
The coreutils should be very reliable. This means that we should never `panic!`.
|
||||
Therefore, you should avoid using `.unwrap()` and `panic!`. Sometimes the use of
|
||||
`unreachable!` can be justified with a comment explaining why that code is
|
||||
unreachable.
|
||||
|
||||
### Don't `exit`
|
||||
|
||||
We want uutils to be embeddable in other programs. This means that no function
|
||||
in uutils should exit the program. Doing so would also lead to code with more
|
||||
confusing control flow. Avoid therefore `std::process::exit` and similar
|
||||
functions which exit the program early.
|
||||
|
||||
### `unsafe`
|
||||
|
||||
uutils cannot be entirely safe, because we have to call out to `libc` and do
|
||||
syscalls. However, we still want to limit our use of `unsafe`. We generally only
|
||||
accept `unsafe` for FFI, with very few exceptions. Note that performance is very
|
||||
rarely a valid argument for using `unsafe`.
|
||||
|
||||
If you still need to write code with `unsafe`, make sure to read the
|
||||
[Rustonomicon](https://doc.rust-lang.org/nomicon/intro.html) and annotate the
|
||||
calls with `// SAFETY:` comments explaining why the use of `unsafe` is sound.
|
||||
|
||||
### Macros
|
||||
|
||||
Macros can be a great tool, but they are also usually hard to understand. They
|
||||
should be used sparingly. Make sure to explore simpler options before you reach
|
||||
for a solution involving macros.
|
||||
|
||||
### `str`, `OsStr` & `Path`
|
||||
|
||||
Rust has many string-like types, and sometimes it's hard to choose the right
|
||||
one. It's tempting to use `str` (and `String`) for everything, but that is not
|
||||
always the right choice for uutils, because we need to support invalid UTF-8,
|
||||
just like the GNU coreutils. For example, paths on Linux might not be valid
|
||||
UTF-8! Whenever we are dealing with paths, we should therefore stick with
|
||||
`OsStr` and `Path`. Make sure that you only convert to `str`/`String` if you
|
||||
know that something is always valid UTF-8. If you need more operations on
|
||||
`OsStr`, you can use the [`bstr`](https://docs.rs/bstr/latest/bstr/) crate.
|
||||
|
||||
### Doc-comments
|
||||
|
||||
We use rustdoc for our documentation, so it's best to follow
|
||||
[rustdoc's guidelines](https://doc.rust-lang.org/rustdoc/how-to-write-documentation.html#documenting-components).
|
||||
Make sure that your documentation is not just repeating the name of the
|
||||
function, but actually giving more useful information. Rustdoc recommends the
|
||||
following structure:
|
||||
|
||||
```
|
||||
[short sentence explaining what it is]
|
||||
|
||||
[more detailed explanation]
|
||||
|
||||
[at least one code example that users can copy/paste to try it]
|
||||
|
||||
[even more advanced explanations if necessary]
|
||||
```
|
||||
|
||||
### Other comments
|
||||
|
||||
Comments should be written to _explain_ the code, not to _describe_ the code.
|
||||
Try to focus on explaining _why_ the code is the way it is. If you feel like you
|
||||
have to describe the code, that's usually a sign that you could improve the
|
||||
naming of variables and functions.
|
||||
|
||||
If you edit a piece of code, make sure to update any comments that need to
|
||||
change as a result. The only thing worse than having no comments is having
|
||||
outdated comments!
|
||||
|
||||
## Git Etiquette
|
||||
|
||||
To ensure easy collaboration, we have guidelines for using Git and GitHub.
|
||||
|
||||
### Commits
|
||||
|
||||
- Make small and atomic commits.
|
||||
- Keep a clean history of commits.
|
||||
- Write informative commit messages.
|
||||
- Annotate your commit message with the component you're editing. For example:
|
||||
`cp: do not overwrite on with -i` or `uucore: add support for FreeBSD`.
|
||||
- Do not unnecessarily move items around in the code. This makes the changes
|
||||
much harder to review. If you do need to move things around, do that in a
|
||||
separate commit.
|
||||
|
||||
### Commit messages
|
||||
|
||||
You can read this section in the Git book to learn how to write good commit
|
||||
messages: https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project.
|
||||
|
||||
In addition, here are a few examples for a summary line when committing to
|
||||
uutils:
|
||||
|
||||
- commit for a single utility
|
||||
|
||||
```
|
||||
nohup: cleanup and refactor
|
||||
```
|
||||
|
||||
- commit for a utility's tests
|
||||
|
||||
```
|
||||
tests/rm: test new feature
|
||||
```
|
||||
|
||||
Beyond changes to an individual utility or its tests, other summary lines for
|
||||
non-utility modules include:
|
||||
|
||||
```
|
||||
README: add help
|
||||
uucore: add new modules
|
||||
uutils: add new utility
|
||||
gitignore: add temporary files
|
||||
```
|
||||
|
||||
### PRs
|
||||
|
||||
- Make the titles of PRs descriptive.
|
||||
- This means describing the problem you solve. For example, do not write
|
||||
`Fix #1234`, but `ls: fix version sort order`.
|
||||
- You can prefix the title with the utility the PR concerns.
|
||||
- Keep PRs small and self-contained. A set of small PRs is much more likely to
|
||||
get merged quickly than one large PR.
|
||||
- Make sure the CI passes (up to intermittently failing tests).
|
||||
- You know your code best, that's why it's best if you can solve merge conflicts
|
||||
on your branch yourself.
|
||||
- It's up to you whether you want to use `git merge main` or
|
||||
`git rebase main`.
|
||||
- Feel free to ask for help with merge conflicts.
|
||||
- You do not need to ping maintainers to request a review, but it's fine to do
|
||||
so if you don't get a response within a few days.
|
||||
|
||||
## Platforms
|
||||
|
||||
We take pride in supporting many operating systems and architectures. Any code
|
||||
you contribute must at least compile without warnings for all platforms in the
|
||||
CI. However, you can use `#[cfg(...)]` attributes to create platform dependent
|
||||
features.
|
||||
|
||||
## Licensing
|
||||
|
||||
uutils is distributed under the terms of the MIT License; see the `LICENSE` file
|
||||
for details. This is a permissive license, which allows the software to be used
|
||||
with few restrictions.
|
||||
|
||||
Copyrights in the uutils project are retained by their contributors, and no
|
||||
copyright assignment is required to contribute.
|
||||
|
||||
If you wish to add or change dependencies as part of a contribution to the
|
||||
project, a tool like `cargo-license` can be used to show their license details.
|
||||
The following types of license are acceptable:
|
||||
|
||||
- MIT License
|
||||
- Dual- or tri-license with an MIT License option ("Apache-2.0 or MIT" is a
|
||||
popular combination)
|
||||
- "MIT equivalent" license (2-clause BSD, 3-clause BSD, ISC)
|
||||
- License less restrictive than the MIT License (CC0 1.0 Universal)
|
||||
- Apache License version 2.0
|
||||
|
||||
Licenses we will not use:
|
||||
|
||||
- An ambiguous license, or no license
|
||||
- Strongly reciprocal licenses (GNU GPL, GNU LGPL)
|
||||
|
||||
If you wish to add a reference but it doesn't meet these requirements, please
|
||||
raise an issue to describe the dependency.
|
||||
Generated
+1276
File diff suppressed because it is too large
Load Diff
+31
@@ -0,0 +1,31 @@
|
||||
[package]
|
||||
name = "uu_grep"
|
||||
description = "A Rust implementation of GNU Grep"
|
||||
repository = "https://github.com/microsoft/uutils-grep"
|
||||
edition = "2024"
|
||||
rust-version = "1.88.0"
|
||||
version = "0.1.0"
|
||||
license = "MIT"
|
||||
homepage = "https://github.com/microsoft/uutils-grep"
|
||||
keywords = ["grep", "uutils", "cross-platform", "cli", "utility"]
|
||||
categories = ["command-line-utilities"]
|
||||
|
||||
[[bin]]
|
||||
name = "grep"
|
||||
path = "src/main.rs"
|
||||
|
||||
[lib]
|
||||
name = "uu_grep"
|
||||
path = "src/lib.rs"
|
||||
|
||||
[dependencies]
|
||||
clap = { version = "4.5", features = ["wrap_help", "cargo", "color"] }
|
||||
glob = "0.3.1"
|
||||
memchr = "2.7.2"
|
||||
onig = { version = "~6.5.1", default-features = false }
|
||||
onig_sys = { version = "*", default-features = false }
|
||||
uucore = "0.8.0"
|
||||
walkdir = "2.5"
|
||||
|
||||
[dev-dependencies]
|
||||
uutests = "0.8.0"
|
||||
@@ -0,0 +1,18 @@
|
||||
Copyright (c) uutils developers
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
this software and associated documentation files (the "Software"), to deal in
|
||||
the Software without restriction, including without limitation the rights to
|
||||
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
||||
the Software, and to permit persons to whom the Software is furnished to do so,
|
||||
subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Grep, now in Rust
|
||||
|
||||
A Rust implementation of [GNU Grep](https://www.gnu.org/software/grep/).
|
||||
This project is an initial release and may contain bugs.
|
||||
|
||||
## Building
|
||||
|
||||
Download Rust at: https://rustup.rs/
|
||||
|
||||
```shell
|
||||
# Check out this repository
|
||||
git clone https://github.com/uutils/grep
|
||||
cd grep
|
||||
|
||||
# Build a release version
|
||||
cargo build --release
|
||||
|
||||
# Run!
|
||||
./target/release/grep --help
|
||||
|
||||
# Run tests (if needed; after making changes)
|
||||
cargo test
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
* Does not take `LANG`, etc., into account for handling file encodings (non-UTF8 matches are treated as binary)
|
||||
|
||||
## Contributing
|
||||
|
||||
To contribute to uutils, please see [CONTRIBUTING](CONTRIBUTING.md).
|
||||
|
||||
## License
|
||||
|
||||
uutils is licensed under the MIT License - see the `LICENSE` file for details
|
||||
|
||||
GNU Grep is licensed under the GPL 3.0 or later.
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
We provide security updates only for the latest released version of `uutils/grep`.
|
||||
Older versions may not receive patches.
|
||||
If you are using a version packaged by your Linux distribution, please check with your distribution maintainers for their update policy.
|
||||
|
||||
---
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
**Do not open public GitHub issues for security vulnerabilities.**
|
||||
This prevents accidental disclosure before a fix is available.
|
||||
|
||||
Instead, please use the following method:
|
||||
|
||||
- **Email:** [sylvestre@debian.org](mailto:Sylvestre@debian.org)
|
||||
- **Encryption (optional):** You may encrypt your report using our PGP key:
|
||||
Fingerprint: B60D B599 4D39 BEC4 D1A9 5CCF 7E65 28DA 752F 1BE1
|
||||
---
|
||||
|
||||
### What to Include in Your Report
|
||||
|
||||
To help us investigate and resolve the issue quickly, please include as much detail as possible:
|
||||
|
||||
- **Type of issue:** e.g. privilege escalation, information disclosure.
|
||||
- **Location in the source:** file path, commit hash, branch, or tag.
|
||||
- **Steps to reproduce:** exact commands, test cases, or scripts.
|
||||
- **Special configuration:** any flags, environment variables, or system setup required.
|
||||
- **Affected systems:** OS/distribution and version(s) where the issue occurs.
|
||||
- **Impact:** your assessment of the potential severity (DoS, RCE, data leak, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Disclosure Policy
|
||||
|
||||
We follow a **Coordinated Vulnerability Disclosure (CVD)** process:
|
||||
|
||||
1. We will acknowledge receipt of your report within **10 days**.
|
||||
2. We will investigate, reproduce, and assess the issue.
|
||||
3. We will provide a timeline for developing and releasing a fix.
|
||||
4. Once a fix is available, we will publish a GitHub Security Advisory.
|
||||
5. You will be credited in the advisory unless you request anonymity.
|
||||
@@ -0,0 +1,102 @@
|
||||
pub struct LineView<'a> {
|
||||
/// Line content (without the terminator).
|
||||
pub line: &'a [u8],
|
||||
/// 1-based line number.
|
||||
pub line_number: u64,
|
||||
/// Byte offset of this line in the input stream.
|
||||
pub byte_offset: u64,
|
||||
/// Whether this line matched (vs. being a context line).
|
||||
pub is_match: bool,
|
||||
/// Match positions within the line (start, end).
|
||||
/// Empty for context lines or for matching lines we don't need to highlight.
|
||||
pub match_positions: &'a [(usize, usize)],
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct BufferedLine {
|
||||
pub line: Vec<u8>,
|
||||
pub line_number: u64,
|
||||
pub byte_offset: u64,
|
||||
}
|
||||
|
||||
impl BufferedLine {
|
||||
pub fn view(&self) -> LineView<'_> {
|
||||
LineView {
|
||||
line: &self.line,
|
||||
line_number: self.line_number,
|
||||
byte_offset: self.byte_offset,
|
||||
is_match: false,
|
||||
match_positions: &[],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A fixed-capacity ring buffer of context lines.
|
||||
///
|
||||
/// TODO: Ideally, this would be integrated into `LineBuffer`, which can then
|
||||
/// provide a more optimized `ContextBuffer` when `mmap()` is available.
|
||||
pub struct ContextBuffer {
|
||||
slots: Vec<BufferedLine>,
|
||||
head: usize,
|
||||
len: usize,
|
||||
}
|
||||
|
||||
impl ContextBuffer {
|
||||
pub fn new(capacity: usize) -> Self {
|
||||
let len = if capacity == 0 {
|
||||
0
|
||||
} else {
|
||||
capacity.next_power_of_two()
|
||||
};
|
||||
Self {
|
||||
slots: vec![
|
||||
BufferedLine {
|
||||
line: Vec::new(),
|
||||
line_number: 0,
|
||||
byte_offset: 0,
|
||||
};
|
||||
len
|
||||
],
|
||||
head: 0,
|
||||
len: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn clear(&mut self) {
|
||||
self.head = 0;
|
||||
self.len = 0;
|
||||
}
|
||||
|
||||
pub fn push(&mut self, line: &[u8], line_number: u64, byte_offset: u64) {
|
||||
debug_assert!(
|
||||
!self.slots.is_empty(),
|
||||
"push on zero-capacity ContextBuffer"
|
||||
);
|
||||
|
||||
let mask = self.slots.len() - 1;
|
||||
let slot = &mut self.slots[self.head & mask];
|
||||
|
||||
slot.line.clear();
|
||||
if slot.line.capacity() / 2 > line.len() {
|
||||
slot.line.shrink_to(line.len());
|
||||
}
|
||||
slot.line.extend_from_slice(line);
|
||||
slot.line_number = line_number;
|
||||
slot.byte_offset = byte_offset;
|
||||
|
||||
self.head = self.head.wrapping_add(1);
|
||||
self.len = (self.len + 1).min(self.slots.len());
|
||||
}
|
||||
|
||||
pub fn drain_iter(&mut self) -> impl Iterator<Item = &BufferedLine> {
|
||||
let len = self.len;
|
||||
let slots = &self.slots[..];
|
||||
let mask = self.slots.len().wrapping_sub(1);
|
||||
let start = self.head.wrapping_sub(len);
|
||||
|
||||
self.head = 0;
|
||||
self.len = 0;
|
||||
|
||||
(0..len).map(move |i| &slots[start.wrapping_add(i) & mask])
|
||||
}
|
||||
}
|
||||
+920
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,109 @@
|
||||
use memchr::memchr;
|
||||
use std::fs::File;
|
||||
use std::io::{self, Read as _};
|
||||
|
||||
pub struct LineBuffer {
|
||||
buffer: Vec<u8>,
|
||||
|
||||
/// Start of the valid data and current (incomplete) line in `buffer`.
|
||||
beg: usize,
|
||||
/// Offset of where the next `memchr` should continue scanning from.
|
||||
/// This helps us avoid re-scanning bytes we've already confirmed don't contain a terminator.
|
||||
scan: usize,
|
||||
/// End of the current valid data in `buffer`.
|
||||
end: usize,
|
||||
|
||||
/// Absolute file offset of `buffer[pos]`.
|
||||
next_line_start: u64,
|
||||
/// Set once a file read has returned EOF.
|
||||
eof: bool,
|
||||
/// The line terminator is typically NUL or LF.
|
||||
line_terminator: u8,
|
||||
}
|
||||
|
||||
impl LineBuffer {
|
||||
pub fn new(line_terminator: u8) -> Self {
|
||||
Self {
|
||||
buffer: vec![0; 128 * 1024],
|
||||
|
||||
beg: 0,
|
||||
scan: 0,
|
||||
end: 0,
|
||||
|
||||
next_line_start: 0,
|
||||
eof: false,
|
||||
line_terminator,
|
||||
}
|
||||
}
|
||||
|
||||
/// Reset the buffer to an empty state.
|
||||
pub fn reset(&mut self) {
|
||||
self.beg = 0;
|
||||
self.scan = 0;
|
||||
self.end = 0;
|
||||
self.next_line_start = 0;
|
||||
self.eof = false;
|
||||
}
|
||||
|
||||
/// Read the next line from the given reader.
|
||||
/// Returns `Ok(None)` if the end of the reader is reached.
|
||||
/// Otherwise, returns `Ok(Some((line, line_start)))`, where `line` is the line read (without the
|
||||
/// `line_terminator`), and `line_start` is the absolute byte offset of the start of the line.
|
||||
pub fn read_line(&mut self, file: &mut File) -> io::Result<Option<(&[u8], u64)>> {
|
||||
if self.eof {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
loop {
|
||||
// Look for a line terminator and if found, yield that line.
|
||||
if let Some(off) = memchr(self.line_terminator, &self.buffer[self.scan..self.end]) {
|
||||
let line_start = self.next_line_start;
|
||||
let beg = self.beg;
|
||||
let end = self.scan + off;
|
||||
let line = &self.buffer[beg..end];
|
||||
self.beg = end + 1;
|
||||
self.scan = self.beg;
|
||||
self.next_line_start += (self.beg - beg) as u64;
|
||||
return Ok(Some((line, line_start)));
|
||||
}
|
||||
|
||||
// `buffer[pos..end]` has no terminator. Remember that for the next scan.
|
||||
self.scan = self.end;
|
||||
|
||||
// Move the partial line to the beginning of the buffer.
|
||||
// The idea is that read() calls will either be very slow, and this doesn't matter,
|
||||
// or they'll be very fast with big chunks and we want to maximize the amount of space per syscall.
|
||||
if self.beg > 0 {
|
||||
self.buffer.copy_within(self.beg..self.end, 0);
|
||||
self.end -= self.beg;
|
||||
self.scan -= self.beg;
|
||||
self.beg = 0;
|
||||
}
|
||||
if self.end == self.buffer.len() {
|
||||
// A single line exceeds the current buffer; grow.
|
||||
self.buffer.resize(self.buffer.len() * 2, 0);
|
||||
}
|
||||
|
||||
// Read more data!
|
||||
let n = loop {
|
||||
match file.read(&mut self.buffer[self.end..]) {
|
||||
Ok(n) => break n,
|
||||
Err(e) if e.kind() == io::ErrorKind::Interrupted => {}
|
||||
Err(e) => return Err(e),
|
||||
}
|
||||
};
|
||||
if n == 0 {
|
||||
// EOF: Yield the last line, if any.
|
||||
return if self.beg == self.end {
|
||||
Ok(None)
|
||||
} else {
|
||||
let line = &self.buffer[self.beg..self.end];
|
||||
let line_start = self.next_line_start;
|
||||
self.eof = true; // shortcut the next call to read_line()
|
||||
Ok(Some((line, line_start)))
|
||||
};
|
||||
}
|
||||
self.end += n;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
uucore::bin!(uu_grep);
|
||||
+271
@@ -0,0 +1,271 @@
|
||||
use crate::{Config, RegexMode};
|
||||
use onig::{EncodedBytes, Regex, RegexOptions, Region, SearchOptions, Syntax, SyntaxBehavior};
|
||||
use onig_sys::{OnigEncCtype_ONIGENC_CTYPE_WORD, OnigEncodingUTF8};
|
||||
use uucore::error::{UResult, USimpleError};
|
||||
|
||||
pub struct Matcher<'a> {
|
||||
config: &'a Config<'a>,
|
||||
patterns: Vec<CompiledPattern>,
|
||||
}
|
||||
|
||||
impl<'a> Matcher<'a> {
|
||||
pub fn compile(config: &'a Config<'a>) -> UResult<Self> {
|
||||
let mut patterns = Vec::with_capacity(config.patterns.len());
|
||||
for raw in config.patterns {
|
||||
patterns.push(CompiledPattern::compile(raw, config)?);
|
||||
}
|
||||
Ok(Self { config, patterns })
|
||||
}
|
||||
|
||||
/// Decide whether `line` matches and return the positions to highlight.
|
||||
pub fn match_line(&self, line: &[u8]) -> Option<Vec<(usize, usize)>> {
|
||||
let mut any_seen = false;
|
||||
let positions: Vec<_> = MatchIter::new(&self.patterns, line)
|
||||
.filter(|&(start, end)| {
|
||||
any_seen = true;
|
||||
// Drop zero-length matches from the output.
|
||||
if start == end {
|
||||
return false;
|
||||
}
|
||||
// Drop matches that don't span the whole line if `-x` was requested.
|
||||
if self.config.line_regexp && !(start == 0 && end == line.len()) {
|
||||
return false;
|
||||
}
|
||||
// Drop matches that aren't word matches if `-w` was requested.
|
||||
if self.config.word_regexp && !Self::is_word_match(line, start, end) {
|
||||
return false;
|
||||
}
|
||||
true
|
||||
})
|
||||
.collect();
|
||||
|
||||
let raw_matched = if self.config.line_regexp || self.config.word_regexp {
|
||||
// -w / -x are authoritative once positions are filtered.
|
||||
!positions.is_empty()
|
||||
} else {
|
||||
any_seen
|
||||
};
|
||||
|
||||
if raw_matched != self.config.invert_match {
|
||||
Some(positions)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Cheap match check that doesn't enumerate positions.
|
||||
pub fn is_match(&self, line: &[u8]) -> Option<Vec<(usize, usize)>> {
|
||||
// `-w` / `-x` need positions to filter, so we fall back to `match_line`.
|
||||
let matched = if self.config.line_regexp || self.config.word_regexp {
|
||||
self.match_line(line).is_some()
|
||||
} else {
|
||||
let raw_matched = self.patterns.iter().any(|p| p.is_match(line));
|
||||
raw_matched != self.config.invert_match
|
||||
};
|
||||
matched.then(Vec::new)
|
||||
}
|
||||
|
||||
/// Word-boundary check `-w`.
|
||||
/// NOTE that `-w` does not check both sides, unlike `\b` in a regex.
|
||||
/// Start/End-of-line count as non-words.
|
||||
fn is_word_match(line: &[u8], start: usize, end: usize) -> bool {
|
||||
// SAFETY: This code uses OnigEncodingType such that it can support other types of encodings in the future.
|
||||
unsafe {
|
||||
let mbc_to_code = OnigEncodingUTF8.mbc_to_code.unwrap_unchecked();
|
||||
let is_code_ctype = OnigEncodingUTF8.is_code_ctype.unwrap_unchecked();
|
||||
let line_end = line.as_ptr().add(line.len());
|
||||
|
||||
if end < line.len() {
|
||||
let cp = mbc_to_code(line.as_ptr().add(end), line_end);
|
||||
if is_code_ctype(cp, OnigEncCtype_ONIGENC_CTYPE_WORD) != 0 {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if start > 0 {
|
||||
let left_adjust = OnigEncodingUTF8.left_adjust_char_head.unwrap_unchecked();
|
||||
let head = left_adjust(line.as_ptr(), line.as_ptr().add(start - 1));
|
||||
let cp = mbc_to_code(head, line_end);
|
||||
if is_code_ctype(cp, OnigEncCtype_ONIGENC_CTYPE_WORD) != 0 {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Streaming k-way merge over compiled patterns
|
||||
struct MatchIter<'a> {
|
||||
cursors: Vec<Cursor<'a>>,
|
||||
/// End of the last emitted match.
|
||||
last_end: usize,
|
||||
}
|
||||
|
||||
impl<'a> MatchIter<'a> {
|
||||
fn new(patterns: &'a [CompiledPattern], line: &'a [u8]) -> Self {
|
||||
Self {
|
||||
cursors: patterns
|
||||
.iter()
|
||||
.map(|pattern| {
|
||||
let mut c = Cursor {
|
||||
pattern,
|
||||
line,
|
||||
offset: 0,
|
||||
pending: None,
|
||||
};
|
||||
c.refill();
|
||||
c
|
||||
})
|
||||
.collect(),
|
||||
last_end: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Iterator for MatchIter<'a> {
|
||||
type Item = (usize, usize);
|
||||
|
||||
fn next(&mut self) -> Option<Self::Item> {
|
||||
// Discard stale pendings that fall before the last emit.
|
||||
for cursor in &mut self.cursors {
|
||||
if matches!(cursor.pending, Some((s, _)) if s < self.last_end) {
|
||||
cursor.offset = self.last_end;
|
||||
cursor.refill();
|
||||
}
|
||||
}
|
||||
|
||||
// Pick the leftmost pending.
|
||||
// Tie-break by largest end so POSIX leftmost-longest holds across
|
||||
// patterns too (e.g. `-e a -e ab` against `ab` emits `ab`).
|
||||
let best_idx = self
|
||||
.cursors
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(i, c)| c.pending.map(|p| (i, p)))
|
||||
.min_by_key(|&(_, (s, e))| (s, std::cmp::Reverse(e)))
|
||||
.map(|(i, _)| i)?;
|
||||
|
||||
let (start, end) = self.cursors[best_idx].pending.unwrap();
|
||||
self.cursors[best_idx].refill();
|
||||
self.last_end = end;
|
||||
Some((start, end))
|
||||
}
|
||||
}
|
||||
|
||||
struct Cursor<'a> {
|
||||
pattern: &'a CompiledPattern,
|
||||
line: &'a [u8],
|
||||
/// Where the next `search_leftmost` call should start.
|
||||
offset: usize,
|
||||
/// Pre-fetched next match for this pattern.
|
||||
/// `None` once the pattern is exhausted.
|
||||
pending: Option<(usize, usize)>,
|
||||
}
|
||||
|
||||
impl Cursor<'_> {
|
||||
fn refill(&mut self) {
|
||||
if self.offset >= self.line.len() {
|
||||
self.pending = None;
|
||||
return;
|
||||
}
|
||||
let Some((start, leftmost_end)) = self.pattern.search_leftmost(self.line, self.offset)
|
||||
else {
|
||||
self.pending = None;
|
||||
return;
|
||||
};
|
||||
let end = self
|
||||
.pattern
|
||||
.longest_end_at(self.line, start)
|
||||
.unwrap_or(leftmost_end);
|
||||
// Advance the next search past the match we just found.
|
||||
// Zero-length matches need a +1 nudge to avoid spinning forever.
|
||||
self.offset = end.max(start + 1);
|
||||
self.pending = Some((start, end));
|
||||
}
|
||||
}
|
||||
|
||||
struct CompiledPattern {
|
||||
/// Default semantics. It's decently fast and used for searching.
|
||||
leftmost: Regex,
|
||||
/// Compiled with `FIND_LONGEST`. If used for a search, it'll search the
|
||||
/// entire haystack to find the longest. This makes it unsuitable for searching,
|
||||
/// but it's perfect for a second, anchored match pass for POSIX semantics.
|
||||
longest_anchored: Regex,
|
||||
}
|
||||
|
||||
impl CompiledPattern {
|
||||
fn compile(pattern: &str, config: &Config) -> UResult<Self> {
|
||||
let mut syntax = *match config.regex_mode {
|
||||
RegexMode::Fixed => Syntax::asis(),
|
||||
RegexMode::Basic => Syntax::grep(),
|
||||
RegexMode::Extended => Syntax::gnu_regex(),
|
||||
RegexMode::Perl => Syntax::perl(),
|
||||
};
|
||||
if !matches!(config.regex_mode, RegexMode::Fixed) {
|
||||
syntax.enable_behavior(SyntaxBehavior::SYNTAX_BEHAVIOR_ALLOW_INTERVAL_LOW_ABBREV);
|
||||
}
|
||||
|
||||
let mut options = RegexOptions::REGEX_OPTION_NONE;
|
||||
if config.ignore_case {
|
||||
options |= RegexOptions::REGEX_OPTION_IGNORECASE;
|
||||
}
|
||||
|
||||
fn compile_with(pattern: &str, syntax: &Syntax, options: RegexOptions) -> UResult<Regex> {
|
||||
Regex::with_options_and_encoding(pattern, options, syntax).map_err(|err| {
|
||||
USimpleError::new(2, format!("invalid pattern \"{pattern}\": {err}"))
|
||||
})
|
||||
}
|
||||
|
||||
let leftmost = compile_with(pattern, &syntax, options)?;
|
||||
let longest_anchored = compile_with(
|
||||
pattern,
|
||||
&syntax,
|
||||
options | RegexOptions::REGEX_OPTION_FIND_LONGEST,
|
||||
)?;
|
||||
Ok(Self {
|
||||
leftmost,
|
||||
longest_anchored,
|
||||
})
|
||||
}
|
||||
|
||||
/// Find the leftmost match starting at or after `offset`.
|
||||
fn search_leftmost(&self, line: &[u8], offset: usize) -> Option<(usize, usize)> {
|
||||
let mut region = Region::new();
|
||||
self.leftmost.search_with_encoding(
|
||||
EncodedBytes::from_parts(line, &raw mut OnigEncodingUTF8),
|
||||
offset,
|
||||
line.len(),
|
||||
SearchOptions::SEARCH_OPTION_NONE,
|
||||
Some(&mut region),
|
||||
)?;
|
||||
region.pos(0)
|
||||
}
|
||||
|
||||
/// Given a known leftmost start `start`, return the longest extent
|
||||
/// of a match anchored exactly there = POSIX leftmost-longest end.
|
||||
fn longest_end_at(&self, line: &[u8], start: usize) -> Option<usize> {
|
||||
let mut region = Region::new();
|
||||
self.longest_anchored.match_with_encoding(
|
||||
EncodedBytes::from_parts(line, &raw mut OnigEncodingUTF8),
|
||||
start,
|
||||
SearchOptions::SEARCH_OPTION_NONE,
|
||||
Some(&mut region),
|
||||
);
|
||||
region.pos(0).map(|(_, end)| end)
|
||||
}
|
||||
|
||||
/// True if any match exists in `line` (including zero-length).
|
||||
fn is_match(&self, line: &[u8]) -> bool {
|
||||
self.leftmost
|
||||
.search_with_encoding(
|
||||
EncodedBytes::from_parts(line, &raw mut OnigEncodingUTF8),
|
||||
0,
|
||||
line.len(),
|
||||
SearchOptions::SEARCH_OPTION_NONE,
|
||||
None,
|
||||
)
|
||||
.is_some()
|
||||
}
|
||||
}
|
||||
+271
@@ -0,0 +1,271 @@
|
||||
use crate::Config;
|
||||
use crate::context_buffer::LineView;
|
||||
use std::ffi::OsStr;
|
||||
use std::io::{self, BufWriter, StdoutLock, Write};
|
||||
use std::path::Path;
|
||||
|
||||
#[cfg(target_pointer_width = "64")]
|
||||
const BUF_SIZE: usize = 128 * 1024;
|
||||
#[cfg(target_pointer_width = "32")]
|
||||
const BUF_SIZE: usize = 16 * 1024;
|
||||
|
||||
pub struct OutputWriter<'a> {
|
||||
config: &'a Config<'a>,
|
||||
out: BufWriter<StdoutLock<'static>>,
|
||||
line_number_width: usize,
|
||||
}
|
||||
|
||||
impl<'a> OutputWriter<'a> {
|
||||
pub fn new(config: &'a Config<'a>) -> Self {
|
||||
Self {
|
||||
config,
|
||||
out: BufWriter::with_capacity(BUF_SIZE, io::stdout().lock()),
|
||||
line_number_width: 0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns true if line numbers will be padded with whitespace (`-T`),
|
||||
/// which means that you should call `set_line_number_width`.
|
||||
pub fn wants_padded_line_numbers(&self) -> bool {
|
||||
self.config.initial_tab && self.config.line_number
|
||||
}
|
||||
|
||||
/// Set the minimum field width for line numbers (used with `-T`).
|
||||
///
|
||||
/// It's sort of wrong for this to be a setter, because it's technically
|
||||
/// per-session state in searcher.rs, but it's convenient and it's cheap.
|
||||
pub fn set_line_number_width(&mut self, width: usize) {
|
||||
debug_assert!(self.wants_padded_line_numbers(), "don't call this function");
|
||||
self.line_number_width = width;
|
||||
}
|
||||
|
||||
/// Flush stdout.
|
||||
pub fn flush(&mut self) -> io::Result<()> {
|
||||
self.out.flush()
|
||||
}
|
||||
|
||||
/// Write a matching or context line.
|
||||
pub fn write_line(&mut self, view: &LineView<'_>, filename: &Path) -> io::Result<()> {
|
||||
if self.config.only_matching && view.is_match {
|
||||
self.write_only_matching(view, filename)
|
||||
} else {
|
||||
self.write_line_with_matches(view, filename)
|
||||
}
|
||||
}
|
||||
|
||||
/// Write only the matching portions of a line (`-o` mode).
|
||||
fn write_only_matching(&mut self, view: &LineView<'_>, filename: &Path) -> io::Result<()> {
|
||||
for &(start, end) in view.match_positions {
|
||||
if start == end {
|
||||
continue; // Skip zero-length matches
|
||||
}
|
||||
|
||||
self.write_prefix(
|
||||
filename,
|
||||
view.line_number,
|
||||
view.byte_offset + start as u64,
|
||||
b':',
|
||||
)?;
|
||||
|
||||
self.write_colored_bytes(
|
||||
self.config.color_config.matched_selected,
|
||||
&view.line[start..end],
|
||||
)?;
|
||||
self.write_terminator()?;
|
||||
}
|
||||
|
||||
self.maybe_flush()
|
||||
}
|
||||
|
||||
/// Otherwise, write the whole line, with optional color highlighting of matches.
|
||||
fn write_line_with_matches(&mut self, view: &LineView<'_>, filename: &Path) -> io::Result<()> {
|
||||
self.write_prefix(
|
||||
filename,
|
||||
view.line_number,
|
||||
view.byte_offset,
|
||||
if view.is_match { b':' } else { b'-' },
|
||||
)?;
|
||||
|
||||
let mut last_end = 0;
|
||||
|
||||
if self.config.use_color && view.is_match && !view.match_positions.is_empty() {
|
||||
for &(start, end) in view.match_positions {
|
||||
if start > last_end {
|
||||
self.out.write_all(&view.line[last_end..start])?;
|
||||
}
|
||||
if start < end {
|
||||
let match_bytes = &view.line[start..end];
|
||||
self.write_colored_bytes(
|
||||
self.config.color_config.matched_selected,
|
||||
match_bytes,
|
||||
)?;
|
||||
}
|
||||
last_end = end;
|
||||
}
|
||||
}
|
||||
|
||||
if last_end < view.line.len() {
|
||||
self.out.write_all(&view.line[last_end..])?;
|
||||
}
|
||||
|
||||
self.write_terminator()?;
|
||||
self.maybe_flush()
|
||||
}
|
||||
|
||||
/// Write a line prefix.
|
||||
fn write_prefix(
|
||||
&mut self,
|
||||
filename: &Path,
|
||||
line_number: u64,
|
||||
byte_offset: u64,
|
||||
sep_char: u8,
|
||||
) -> io::Result<()> {
|
||||
if self.config.show_filename {
|
||||
self.write_colored_fmt(
|
||||
self.config.color_config.filename,
|
||||
format_args!("{}", filename.display()),
|
||||
)?;
|
||||
if self.config.null_separator {
|
||||
self.out.write_all(b"\0")?;
|
||||
} else {
|
||||
self.write_separator(sep_char)?;
|
||||
}
|
||||
}
|
||||
|
||||
if self.config.line_number {
|
||||
let width = self.line_number_width;
|
||||
self.write_colored_fmt(
|
||||
self.config.color_config.line_number,
|
||||
format_args!("{:>width$}", line_number, width = width),
|
||||
)?;
|
||||
self.write_separator(sep_char)?;
|
||||
}
|
||||
|
||||
if self.config.byte_offset {
|
||||
self.write_colored_fmt(
|
||||
self.config.color_config.byte_offset,
|
||||
format_args!("{}", byte_offset),
|
||||
)?;
|
||||
self.write_separator(sep_char)?;
|
||||
}
|
||||
|
||||
if self.config.initial_tab
|
||||
&& (self.config.line_number || self.config.byte_offset || self.config.show_filename)
|
||||
{
|
||||
self.out.write_all(b"\t")?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Write the count line for `-c` mode.
|
||||
pub fn write_count(&mut self, count: u64, filename: &Path) -> io::Result<()> {
|
||||
if self.config.show_filename {
|
||||
let sep = if self.config.null_separator {
|
||||
b'\0'
|
||||
} else {
|
||||
b':'
|
||||
};
|
||||
write!(self.out, "{}{}", filename.display(), sep as char)?;
|
||||
}
|
||||
|
||||
writeln!(self.out, "{}", count)?;
|
||||
self.maybe_flush()
|
||||
}
|
||||
|
||||
/// Write filename for `-l` / `-L` mode.
|
||||
pub fn write_filename(&mut self, path: &Path) -> io::Result<()> {
|
||||
self.write_colored_fmt(
|
||||
self.config.color_config.filename,
|
||||
format_args!("{}", path.display()),
|
||||
)?;
|
||||
self.out.write_all(if self.config.null_separator {
|
||||
b"\0"
|
||||
} else {
|
||||
b"\n"
|
||||
})?;
|
||||
self.maybe_flush()
|
||||
}
|
||||
|
||||
/// Write an IO error to stderr.
|
||||
pub fn report_io_error(&self, label: &OsStr, err: &io::Error) {
|
||||
if !self.config.no_messages && !self.config.quiet {
|
||||
eprintln!("grep: {label}: {err}", label = label.to_string_lossy());
|
||||
}
|
||||
}
|
||||
|
||||
/// Write the "binary file matches" message to stderr.
|
||||
pub fn report_binary_match(&self, path: &Path) {
|
||||
eprintln!("grep: {}: binary file matches", path.display());
|
||||
}
|
||||
|
||||
/// Write the group separator between context groups.
|
||||
pub fn write_group_separator(&mut self) -> io::Result<()> {
|
||||
if let Some(sep) = self.config.group_separator {
|
||||
self.write_colored_bytes(self.config.color_config.separator, sep.as_bytes())?;
|
||||
self.out.write_all(b"\n")?;
|
||||
self.maybe_flush()
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Flush in `--line-buffered` mode.
|
||||
fn maybe_flush(&mut self) -> io::Result<()> {
|
||||
if self.config.line_buffered {
|
||||
self.out.flush()
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Write the line/record terminator (`\n`, or `\0` under `-z`).
|
||||
fn write_terminator(&mut self) -> io::Result<()> {
|
||||
self.out
|
||||
.write_all(if self.config.null_data { b"\0" } else { b"\n" })
|
||||
}
|
||||
|
||||
fn write_separator(&mut self, ch: u8) -> io::Result<()> {
|
||||
self.write_colored_bytes(self.config.color_config.separator, &[ch])
|
||||
}
|
||||
|
||||
fn write_colored_bytes(&mut self, color: &str, text: &[u8]) -> io::Result<()> {
|
||||
let colored = self.config.use_color && !color.is_empty();
|
||||
if colored {
|
||||
self.write_colored_prefix(color)?;
|
||||
}
|
||||
self.out.write_all(text)?;
|
||||
if colored {
|
||||
self.write_colored_suffix()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_colored_fmt(&mut self, color: &str, args: std::fmt::Arguments<'_>) -> io::Result<()> {
|
||||
let colored = self.config.use_color && !color.is_empty();
|
||||
if colored {
|
||||
self.write_colored_prefix(color)?;
|
||||
}
|
||||
self.out.write_fmt(args)?;
|
||||
if colored {
|
||||
self.write_colored_suffix()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_colored_prefix(&mut self, color: &str) -> io::Result<()> {
|
||||
write!(self.out, "\x1b[{}m", color)?;
|
||||
if !self.config.color_config.no_erase {
|
||||
self.out.write_all(b"\x1b[K")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_colored_suffix(&mut self) -> io::Result<()> {
|
||||
if self.config.color_config.no_erase {
|
||||
self.out.write_all(b"\x1b[m")
|
||||
} else {
|
||||
self.out.write_all(b"\x1b[m\x1b[K")
|
||||
}
|
||||
}
|
||||
}
|
||||
+467
@@ -0,0 +1,467 @@
|
||||
use crate::context_buffer::{ContextBuffer, LineView};
|
||||
use crate::line_buffer::LineBuffer;
|
||||
use crate::matcher::Matcher;
|
||||
use crate::output::OutputWriter;
|
||||
use crate::{BinaryMode, Config, DeviceMode, DirectoryMode};
|
||||
use memchr::memchr;
|
||||
use std::ffi::OsStr;
|
||||
use std::fs::File;
|
||||
use std::io;
|
||||
use std::mem::ManuallyDrop;
|
||||
use std::ops::ControlFlow;
|
||||
use std::path::Path;
|
||||
use uucore::error::{ExitCode, FromIo, UResult};
|
||||
use walkdir::WalkDir;
|
||||
|
||||
pub struct Searcher<'a> {
|
||||
config: &'a Config<'a>,
|
||||
writer: OutputWriter<'a>,
|
||||
matcher: Matcher<'a>,
|
||||
any_match: bool,
|
||||
had_error: bool,
|
||||
binary_notice_enabled: bool,
|
||||
|
||||
// Per-session state
|
||||
session_context_buf: ContextBuffer,
|
||||
session_match_count: u64,
|
||||
session_after_remaining: usize,
|
||||
session_last_printed_line: u64, // 0 = nothing yet
|
||||
session_binary_detected: bool,
|
||||
}
|
||||
|
||||
impl<'a> Searcher<'a> {
|
||||
pub fn new(config: &'a Config<'a>, matcher: Matcher<'a>, writer: OutputWriter<'a>) -> Self {
|
||||
Self {
|
||||
config,
|
||||
writer,
|
||||
matcher,
|
||||
any_match: false,
|
||||
had_error: false,
|
||||
binary_notice_enabled: config.binary_mode == BinaryMode::Binary
|
||||
&& !config.quiet
|
||||
&& !config.count
|
||||
&& !config.files_with_matches
|
||||
&& !config.files_without_match,
|
||||
|
||||
session_context_buf: ContextBuffer::new(config.before_context),
|
||||
session_match_count: 0,
|
||||
session_after_remaining: 0,
|
||||
session_last_printed_line: 0,
|
||||
session_binary_detected: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Search the given path.
|
||||
pub fn process_path(&mut self, lb: &mut LineBuffer, path: &Path) -> ControlFlow<()> {
|
||||
if path.is_dir() {
|
||||
match self.config.directory_mode {
|
||||
DirectoryMode::Skip => ControlFlow::Continue(()),
|
||||
// Yield the directory path itself; `process_file` will
|
||||
// report the OS error from trying to open it as a file.
|
||||
DirectoryMode::Read => self.process_file(lb, path),
|
||||
DirectoryMode::Recurse => self.process_dir(lb, path, false),
|
||||
}
|
||||
} else {
|
||||
// Top-level FIFOs/sockets/devices: read by default, drop only on `-D skip`.
|
||||
if self.config.device_mode == DeviceMode::Skip && Self::is_special_file(path) {
|
||||
return ControlFlow::Continue(());
|
||||
}
|
||||
self.process_file(lb, path)
|
||||
}
|
||||
}
|
||||
|
||||
/// Recursive search of the current directory. This is used when `-r`
|
||||
/// is used without a path. In that case, GNU grep emits bare paths
|
||||
/// for which we need special handling.
|
||||
pub fn process_implicit_cwd(&mut self, lb: &mut LineBuffer) -> ControlFlow<()> {
|
||||
self.process_dir(lb, Path::new("."), true)
|
||||
}
|
||||
|
||||
/// Search on standard input.
|
||||
pub fn process_stdin(&mut self, lb: &mut LineBuffer) -> ControlFlow<()> {
|
||||
// Turn the Stdin struct into a File to avoid monomorphization just for files VS stdin.
|
||||
// The cast is "unsafe" because we package a borrowed handle into an owned File.
|
||||
// SAFETY: To make that safe, we simply use ManuallyDrop.
|
||||
#[cfg(windows)]
|
||||
let mut stdin = {
|
||||
use std::os::windows::io::{AsRawHandle, FromRawHandle};
|
||||
let handle = io::stdin().as_raw_handle();
|
||||
ManuallyDrop::new(unsafe { File::from_raw_handle(handle) })
|
||||
};
|
||||
#[cfg(not(windows))]
|
||||
let mut stdin = {
|
||||
use std::os::fd::FromRawFd;
|
||||
ManuallyDrop::new(unsafe { File::from_raw_fd(0) })
|
||||
};
|
||||
|
||||
let path = Path::new(&self.config.label);
|
||||
|
||||
// Stdin is not seekable; use fixed width.
|
||||
if self.writer.wants_padded_line_numbers() {
|
||||
self.writer.set_line_number_width(19);
|
||||
}
|
||||
|
||||
let result = self.session_run(lb, path, &mut stdin);
|
||||
self.record_result(OsStr::new(&self.config.label), result)
|
||||
}
|
||||
|
||||
/// Flush output and produce the overall result.
|
||||
/// Returns true if any input matched.
|
||||
pub fn finish(mut self) -> UResult<()> {
|
||||
self.writer
|
||||
.flush()
|
||||
.map_err_context(|| "(standard output)".to_string())?;
|
||||
|
||||
if self.had_error {
|
||||
Err(ExitCode::new(2))
|
||||
} else if self.any_match {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(ExitCode::new(1)) // aka: no match
|
||||
}
|
||||
}
|
||||
|
||||
fn process_dir(
|
||||
&mut self,
|
||||
lb: &mut LineBuffer,
|
||||
start: &Path,
|
||||
strip_root: bool,
|
||||
) -> ControlFlow<()> {
|
||||
let mut walker = WalkDir::new(start)
|
||||
.follow_links(self.config.follow_symlinks)
|
||||
.into_iter();
|
||||
|
||||
while let Some(entry) = walker.next() {
|
||||
let Ok(entry) = entry else {
|
||||
continue;
|
||||
};
|
||||
|
||||
let file_type = entry.file_type();
|
||||
|
||||
// We're only interested in files, so skip dirs.
|
||||
// If we have a --exclude-dir pattern, skip matching directories entirely.
|
||||
if file_type.is_dir() {
|
||||
if self.config.exclude_dir_globs.matches(entry.file_name()) {
|
||||
walker.skip_current_dir();
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// GNU `-r` doesn't follow symlinks.
|
||||
// With `-R` we already had walkdir resolve them.
|
||||
if file_type.is_symlink() {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Skip non-regular files unless `-D read` was given explicitly.
|
||||
if !file_type.is_file() && self.config.device_mode != DeviceMode::Read {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Handle include/exclude globs.
|
||||
let name = entry.file_name();
|
||||
if (!self.config.include_globs.is_empty() && !self.config.include_globs.matches(name))
|
||||
|| self.config.exclude_globs.matches(name)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
let mut path = entry.path();
|
||||
if strip_root {
|
||||
path = Self::strip_dot_prefix(path);
|
||||
}
|
||||
|
||||
self.process_file(lb, path)?;
|
||||
}
|
||||
|
||||
ControlFlow::Continue(())
|
||||
}
|
||||
|
||||
fn process_file(&mut self, lb: &mut LineBuffer, path: &Path) -> ControlFlow<()> {
|
||||
let result = File::open(path).and_then(|mut file| {
|
||||
if self.writer.wants_padded_line_numbers() {
|
||||
let file_size = file.metadata().map_or(0, |m| m.len());
|
||||
self.writer
|
||||
.set_line_number_width(file_size.max(1).ilog10() as usize + 1);
|
||||
}
|
||||
self.session_run(lb, path, &mut file)
|
||||
});
|
||||
self.record_result(path.as_os_str(), result)
|
||||
}
|
||||
|
||||
fn record_result(&mut self, label: &OsStr, result: io::Result<bool>) -> ControlFlow<()> {
|
||||
match result {
|
||||
Ok(true) => {
|
||||
self.any_match = true;
|
||||
|
||||
// In quiet mode, all we want is the exit code, which means
|
||||
// we can stop searching as soon as we see the first match.
|
||||
if self.config.quiet {
|
||||
return ControlFlow::Break(());
|
||||
}
|
||||
}
|
||||
Ok(false) => {}
|
||||
Err(err) => {
|
||||
self.had_error = true;
|
||||
self.writer.report_io_error(label, &err);
|
||||
}
|
||||
}
|
||||
|
||||
ControlFlow::Continue(())
|
||||
}
|
||||
|
||||
fn session_any_match(&self) -> bool {
|
||||
self.session_match_count > 0
|
||||
}
|
||||
|
||||
fn session_can_match(&self) -> bool {
|
||||
self.config
|
||||
.max_count
|
||||
.is_none_or(|max| self.session_match_count < max)
|
||||
}
|
||||
|
||||
fn session_should_continue(&self) -> bool {
|
||||
self.session_can_match() || self.session_after_remaining > 0
|
||||
}
|
||||
|
||||
fn session_suppress_normal_output(&self) -> bool {
|
||||
self.config.count || self.config.files_without_match || self.session_binary_detected
|
||||
}
|
||||
|
||||
fn session_needs_match_positions(&self) -> bool {
|
||||
(self.config.word_regexp
|
||||
|| self.config.line_regexp
|
||||
|| self.config.only_matching
|
||||
|| self.config.use_color)
|
||||
&& !self.session_suppress_normal_output()
|
||||
}
|
||||
|
||||
/// Should the trailing `Binary file ... matches` notice be emitted?
|
||||
/// Suppressed by `-c`, `-l`, `-L`, `-q` (all folded into
|
||||
/// [`Self::binary_notice_enabled`] at construction time).
|
||||
fn session_should_emit_binary_notice(&self) -> bool {
|
||||
self.binary_notice_enabled && self.session_binary_detected && self.session_any_match()
|
||||
}
|
||||
|
||||
fn session_run(
|
||||
&mut self,
|
||||
lb: &mut LineBuffer,
|
||||
path: &Path,
|
||||
reader: &mut File,
|
||||
) -> io::Result<bool> {
|
||||
// Reset all session (per-file) state.
|
||||
self.session_context_buf.clear();
|
||||
self.session_match_count = 0;
|
||||
self.session_after_remaining = 0;
|
||||
self.session_last_printed_line = 0;
|
||||
self.session_binary_detected = false;
|
||||
lb.reset();
|
||||
|
||||
let mut line_number: u64 = 0;
|
||||
|
||||
while let Some((line, line_start)) = lb.read_line(reader)? {
|
||||
line_number += 1;
|
||||
|
||||
// Handle `-U, --binary` On Windows.
|
||||
#[cfg(windows)]
|
||||
let line =
|
||||
if self.config.strip_cr && !self.config.null_data && line.last() == Some(&b'\r') {
|
||||
&line[..line.len() - 1]
|
||||
} else {
|
||||
line
|
||||
};
|
||||
|
||||
// Any null byte flips us into binary mode.
|
||||
if !self.session_mark_binary_if(|| memchr(0, line).is_some()) {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
if let Some(positions) = self.session_match_line(line) {
|
||||
// TODO: GNU grep respects LANG. Here, I'm always checking for valid UTF-8.
|
||||
if !self.session_mark_binary_if(|| std::str::from_utf8(line).is_err()) {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
// Print the match and context, and update session state accordingly.
|
||||
if !self.session_handle_match(path, line_number, line_start, line, &positions)? {
|
||||
return Ok(true);
|
||||
}
|
||||
|
||||
if self.session_should_emit_binary_notice() {
|
||||
self.writer.report_binary_match(path);
|
||||
return Ok(true);
|
||||
}
|
||||
} else {
|
||||
self.session_handle_nonmatch(path, line_number, line_start, line)?;
|
||||
}
|
||||
|
||||
if !self.session_should_continue() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
self.session_finalize(path)
|
||||
}
|
||||
|
||||
/// Mark the file as binary when `predicate` returns true.
|
||||
#[inline(always)]
|
||||
fn session_mark_binary_if(&mut self, predicate: impl FnOnce() -> bool) -> bool {
|
||||
if self.session_binary_detected || self.config.binary_mode == BinaryMode::Text {
|
||||
return true;
|
||||
}
|
||||
if !predicate() {
|
||||
return true;
|
||||
}
|
||||
self.session_binary_detected = true;
|
||||
self.config.binary_mode != BinaryMode::WithoutMatch
|
||||
}
|
||||
|
||||
fn session_match_line(&self, line: &[u8]) -> Option<Vec<(usize, usize)>> {
|
||||
if !self.session_can_match() {
|
||||
None
|
||||
} else if self.session_needs_match_positions() {
|
||||
self.matcher.match_line(line)
|
||||
} else {
|
||||
self.matcher.is_match(line)
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns false if this file's search is complete (e.g. `-q`, `-l`, `-L`).
|
||||
fn session_handle_match(
|
||||
&mut self,
|
||||
path: &Path,
|
||||
line_number: u64,
|
||||
byte_offset: u64,
|
||||
line: &[u8],
|
||||
positions: &[(usize, usize)],
|
||||
) -> io::Result<bool> {
|
||||
self.session_match_count += 1;
|
||||
|
||||
if self.config.quiet {
|
||||
return Ok(false);
|
||||
}
|
||||
if self.config.files_with_matches {
|
||||
self.writer.write_filename(path)?;
|
||||
return Ok(false);
|
||||
}
|
||||
if self.config.files_without_match {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
if !self.session_suppress_normal_output() {
|
||||
self.session_write_match_with_context(
|
||||
path,
|
||||
&LineView {
|
||||
line,
|
||||
line_number,
|
||||
byte_offset,
|
||||
is_match: true,
|
||||
match_positions: positions,
|
||||
},
|
||||
)?;
|
||||
}
|
||||
|
||||
self.session_after_remaining = self.config.after_context;
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
fn session_handle_nonmatch(
|
||||
&mut self,
|
||||
path: &Path,
|
||||
line_number: u64,
|
||||
byte_offset: u64,
|
||||
line: &[u8],
|
||||
) -> io::Result<()> {
|
||||
if self.session_after_remaining > 0 {
|
||||
if !self.session_suppress_normal_output() {
|
||||
self.writer.write_line(
|
||||
&LineView {
|
||||
line,
|
||||
line_number,
|
||||
byte_offset,
|
||||
is_match: false,
|
||||
match_positions: &[],
|
||||
},
|
||||
path,
|
||||
)?;
|
||||
self.session_last_printed_line = line_number;
|
||||
}
|
||||
self.session_after_remaining -= 1;
|
||||
} else if self.config.before_context > 0
|
||||
&& self.session_can_match()
|
||||
&& !self.session_suppress_normal_output()
|
||||
{
|
||||
self.session_context_buf
|
||||
.push(line, line_number, byte_offset);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn session_write_match_with_context(
|
||||
&mut self,
|
||||
path: &Path,
|
||||
view: &LineView<'_>,
|
||||
) -> io::Result<()> {
|
||||
// Group separator between non-adjacent groups.
|
||||
// `last_printed_line == 0` means we haven't printed anything yet.
|
||||
// = first group = skip the separator
|
||||
if self.config.has_context
|
||||
&& self.session_last_printed_line > 0
|
||||
&& view.line_number > self.session_last_printed_line + 1
|
||||
{
|
||||
self.writer.write_group_separator()?;
|
||||
}
|
||||
|
||||
for ctx in self.session_context_buf.drain_iter() {
|
||||
if ctx.line_number > self.session_last_printed_line {
|
||||
self.writer.write_line(&ctx.view(), path)?;
|
||||
self.session_last_printed_line = ctx.line_number;
|
||||
}
|
||||
}
|
||||
|
||||
self.writer.write_line(view, path)?;
|
||||
self.session_last_printed_line = view.line_number;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// End-of-file bookkeeping: count / `-L` / binary notice.
|
||||
fn session_finalize(&mut self, path: &Path) -> io::Result<bool> {
|
||||
if self.config.count && !self.config.files_with_matches && !self.config.files_without_match
|
||||
{
|
||||
self.writer.write_count(self.session_match_count, path)?;
|
||||
}
|
||||
if self.config.files_without_match && !self.session_any_match() {
|
||||
self.writer.write_filename(path)?;
|
||||
}
|
||||
if self.session_should_emit_binary_notice() {
|
||||
self.writer.report_binary_match(path);
|
||||
}
|
||||
Ok(self.session_any_match())
|
||||
}
|
||||
|
||||
fn strip_dot_prefix(path: &Path) -> &Path {
|
||||
let bytes = path.as_os_str().as_encoded_bytes();
|
||||
|
||||
#[cfg(windows)]
|
||||
let bytes = bytes
|
||||
.strip_prefix(b".\\")
|
||||
.or_else(|| bytes.strip_prefix(b"./"))
|
||||
.unwrap_or(bytes);
|
||||
|
||||
#[cfg(not(windows))]
|
||||
let bytes = bytes.strip_prefix(b"./").unwrap_or(bytes);
|
||||
|
||||
// SAFETY: We sliced off a pure ASCII prefix off of `path`.
|
||||
Path::new(unsafe { OsStr::from_encoded_bytes_unchecked(bytes) })
|
||||
}
|
||||
|
||||
fn is_special_file(path: &Path) -> bool {
|
||||
match std::fs::metadata(path) {
|
||||
Ok(m) => {
|
||||
let ft = m.file_type();
|
||||
!ft.is_file() && !ft.is_dir()
|
||||
}
|
||||
Err(_) => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
+1143
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user