mirror of
https://github.com/encounter/wasmtime.git
synced 2026-03-30 11:42:15 -07:00
main
103 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nick Fitzgerald
|
1d8a0983bc |
Split out TypeRegistry's open-coded slab arena into into a reusable type (#7986)
First of all, it is just a nice separation of concerns. Second of all, as I design the GC rooting APIs for Wasmtime's upcoming Wasm GC support, I want this same thing and I'd rather not open code it multiple times. |
||
 Alex Crichton
|
51ed135013 |
Remove the WASI submodule (#7984)
* Remove the `WASI` submodule Historically the `WASI` submodule in this repository was used to pull the sources of the `*.witx` files for preview1. This has never been used by the `wasmtime-wasi` crate (which has its own copy in the `crates/wasi/witx` folder) and was only ever used by `wasi-common`. This submodule also served as a location for the `witx` crate itself. Neither of these are really needed any more as the `*.witx` files are unlikely to ever really change again. This commit removes the submodule entirely, as well as the `path` dependency on `witx`, and copies the `*.witx` files in the same manner as the `wasmtime-wasi` crate. * Updated vet entries for witx * Remove witx special cases in publish script * Change how the wasi-common package is built |
||
 Pat Hickey
|
2b00a541f3 |
Make wasi-common self-contained, deprecate exports from wasmtime-wasi (#7881)
* WIP: try to make wasi-common self contained. * rebase: cargo.lock * remove all dependencies between wasi-common and wasmtime-wasi * use wasi-common directly throughout tests, benches, examples, cli run * wasi-threads: use wasi-common's maybe_exit_on_error in spawned thread not a very modular design, but at this point wasi-common and wasi-threads are forever wed * fix wasmtime's docs * re-introduce wasmtime-wasi's exports of wasi-common definitions behind deprecated * factor out determining i32 process exit code and remove libc dep because rustix provides the same constant * commands/run: inline the logic about aborting on trap since this is the sole place in the codebase its used * Add high-level summary to wasi-common's top-level doc comment. * c-api: fix use of wasi_cap_std_sync => wasi_common::sync, wasmtime_wasi => wasi_common * fix tokio example * think better of combining downcast and masking into one method * fix references to wasmtime_wasi in docs prtest:full * benches: use wasi-common * cfg-if around use of rustix::process because that doesnt exist on windows * wasi-common: include tests, caught by verify-publish * fix another bench * exit requires wasmtime dep. caught by verify-publish. |
||
 Max Brunsfeld
|
825494fe6a |
Add wasmtime-c-api-impl to the list of crates to publish (#7837)
* Add wasmtime-c-api-impl to the list of crates to publish * Enable rustdoc and publishing for c-api crate * Provide paths to c-api headers as cargo links metadata * Add a README section about using wasm-c-api in a rust crate * In C API doc comment, mention use case for crates w/ C bindings * Enable publishing for wasmtime-c-api-macros (prtest:full) * Move c-api crates later in the publishing sequence (prtest:full) |
||
 Adam Bratschi-Kaye
|
2fcf41f05b |
Remove wasmtime-jit (#7769)
* Move `jit` crate to `environ` Move the platform agnostic parts of the crate `wasmtime-jit` to `wasmtime-environ`. This is the first part of the refactoring discussed here: https://github.com/bytecodealliance/wasmtime/issues/7652 and a follow up will move the remaining parts of `wasmtime-jit` so that the crate can be deleted. * Move `jit` crate to `wasmtime` Move the remaining parts of `wasmtime-jit` to the `wasmtime` crate and remove `wasmtime-jit`. This is part of the refactoring discussed in https://github.com/bytecodealliance/wasmtime/issues/7652. * undo toml formatting * Trigger pipeline: prtest:full * Remove `jit` directory * move `ProfilingAgent` out of `profiling` feature * add links to ELF_NAME_DATA |
||
 ssunkin-fastly
|
ca5a9db0d1 |
Memcheck for Wasm guests in Wasmtime (#6820)
* attempt at inserting things where i think they might belong + questions * entry hook + questions * commented out all changes, doc comment errors * fix doc comment * libcalls build now!!!! * initial check_malloc_exit setup * WIP: load/store hooks * hooks added + building * added valgrind library * made wasm-valgrind accessible in wasmtime * check_malloc filled in... * move valgrind_state to an appropriate part of instance it works!!!!! * yay it's working! (?) i think?? * stack tracing in progress * errors + num bytes displayed * initial valgrind configuration * valgrind conditional some warnings fixed * conditional compilation + CLI flag finished * panic!() changed to bail!() * started adding doc comments * added memory grow hook + fixed access size handling * removed test.wasm * removed malloc_twice.wat * doc comments in spec.rs * pr feedback addressed * ran cargo fmt * addressing more feedback * Remove fuzz crate from wmemcheck. * Review feedback and test fix. * add wasmtime-wmemcheck crate to publish allowlist. * fix build without compiler features * reorder crates in publish list * Add trampolines for libcalls on s390x. * Make wasmtime-wmemcheck dep an exact version requirement. --------- Co-authored-by: iximeow <awortman@fastly.com> Co-authored-by: Chris Fallin <chris@cfallin.org> Co-authored-by: iximeow <git@iximeow.net> |
||
|
Alex Crichton
|
2897e409db |
Remove the implementation of wasi-crypto (#6816)
* Remove the implementation of wasi-crypto This commit is a follow-up to the discussion on #6732. This removes Wasmtime's implementation of the wasi-crypto proposal from in-tree along with its various support in CI, configuration, etc. See the discussion on #6732 for the full information but at a high level the main reasons for removing the implementation at this time are: * There is not currently an active maintainer of the Wasmtime integration here for wasi-crypto. * There are known issues with the code quality of the implementation such as transmutes of guest-owned memory to `&'static mut [u8]` and known unsafety in dependencies. * The size and breadth of the dependency tree brings maintenance burden and overhead to managing Wasmtime's dependency tree. As mentioned on the issue this commit does not mean that Wasmtime doesn't want to implement the wasi-crypto proposal. Instead the "tier 3" status of wasi-crypto needs to be re-attained to be included back in-tree, which would mean resolving the above issues. Note that this commit is intentionally just after the 13.0.0 branch point which means that this is slated for Wasmtime 14 to be released on September 20. * Remove some cfgs * Remove wasi-crypto CI |
||
 Adam Petro
|
ca90650c12 |
Support multiple versions of wasmtime in the same crate (#6673)
* Attempt versioned exports to facilitate having multiple versions in the same crate * Modify approach to use `export_name` and `link_name` * Only apply version to names in assembly and foreign item fns * Attempt to handle the s390x case * Fix alignment of backslashes in assembly file * Pretend I understand the preprocessor * Version symbols in `crates/runtime/src/helpers.c` * Stop versioning `__jit_debug_register_code` because gdb relies on it and it is uses `weak` linkage * Version symbol in `crates/fiber/src/windows.c` * Consolidate `LitStr` creation in macro * Add new crate to publish script and supply-chain config * Fix order in supply chain config * Set `audit-as-crates-io` to false * Missing `versioned_link` for Windows * Version strings used in debug * Formatting * Get rid of `versioned_str` and bring back `versioned_suffix` --------- Co-authored-by: Alex Crichton <alex@alexcrichton.com> |
||
 Saúl Cabrera
|
afde47c214 |
winch: Drop FuncEnv trait (#6443)
This commit is a small cleanup to drop the usage of the `FuncEnv` trait. In https://github.com/bytecodealliance/wasmtime/pull/6358, we agreed on making `winch-codegen` directly depend on `wasmtime-environ`. Introducing a direct relatioship between `winch-codegen` and `wasmtime-environ` means that the `FuncEnv` trait is no longer serving its original purpose, and we can drop the usage of the trait and use the types exposed from `winch-codegen` directly instead. Even though this change drops the `FuncEnv` trait, it still keeps a `FuncEnv` struct, which is used during code generation. |
||
|
Saúl Cabrera
|
20c5836295 |
winch: Implement new trampolines (#6358)
* winch: Implement new trampolines This change is a follow-up to https://github.com/bytecodealliance/wasmtime/pull/6262, in which the new trampolines, described [here](https://github.com/bytecodealliance/rfcs/blob/main/accepted/tail-calls.md#new-trampolines-and-vmcallercheckedanyfunc-changes), were introduced to Wasmtime. This change, focuses on the `array-to-wasm`, `native-to-wasm` and `wasm-to-native` trampolines to restore Winch's working state prior to the introduction of the new trampolines. It's worth noting that the new approach for trampolines make it easier to support the `TypedFunc` API in Winch. Prior to the introduction of the new trampolines, it was not obvious how to approach it. This change also introduces a pinned register that will hold the `VMContext` pointer, which is loaded in the `*-to-wasm` trampolines; the `VMContext` register is a pre-requisite to this change to support the `wasm-to-native` trampolines. Lastly, with the introduction of the `VMContext` register and the `wasm-to-native` trampolines, this change also introduces support for calling function imports, which is a variation of the already existing calls to locally defined functions. The other notable piece of this change aside from the trampolines is `winch-codegen`'s dependency on `wasmtime-environ`. Winch is so closely tied to the concepts exposed by the wasmtime crates that it makes sense to tie them together, even though the separation provides some advantages like easier testing in some cases, in the long run, there's probably going to be less need to test Winch in isolation and rather we'd rely more on integration style tests which require all of Wasmtime pieces anyway (fuzzing, spec tests, etc). This change doesn't update the existing implmenetation of `winch_codegen::FuncEnv`, but the intention is to update that part after this change. prtest:full * tests: Ignore miri in Winch integration tests * Remove hardcoded alignment and addend |
||
 Brendan Burns
|
2d34dbef4b |
Begin implementation of wasi-http (#5929)
* Integrate experimental HTTP into wasmtime. * Reset Cargo.lock * Switch to bail!, plumb options partially. * Implement timeouts. * Remove generated files & wasm, add Makefile * Remove generated code textfile * Update crates/wasi-http/Cargo.toml Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com> * Update crates/wasi-http/Cargo.toml Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com> * Extract streams from request/response. * Fix read for len < buffer length. * Formatting. * types impl: swap todos for traps * streams_impl: idioms, and swap todos for traps * component impl: idioms, swap all unwraps for traps, swap all todos for traps * http impl: idiom * Remove an unnecessary mut. * Remove an unsupported function. * Switch to the tokio runtime for the HTTP request. * Add a rust example. * Update to latest wit definition * Remove example code. * wip: start writing a http test... * finish writing the outbound request example havent executed it yet * better debug output * wasi-http: some stubs required for rust rewrite of the example * add wasi_http tests to test-programs * CI: run the http tests * Fix some warnings. * bump new deps to latest releases (#3) * Add tests for wasi-http to test-programs (#2) * wip: start writing a http test... * finish writing the outbound request example havent executed it yet * better debug output * wasi-http: some stubs required for rust rewrite of the example * add wasi_http tests to test-programs * CI: run the http tests * bump new deps to latest releases h2 0.3.16 http 0.2.9 mio 0.8.6 openssl 0.10.48 openssl-sys 0.9.83 tokio 1.26.0 --------- Co-authored-by: Brendan Burns <bburns@microsoft.com> * Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs * Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs * Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs * wasi-http: fix cargo.toml file and publish script to work together (#4) unfortunately, the publish script doesn't use a proper toml parser (in order to not have any dependencies), so the whitespace has to be the trivial expected case. then, add wasi-http to the list of crates to publish. * Update crates/test-programs/build.rs * Switch to rustls * Cleanups. * Merge switch to rustls. * Formatting * Remove libssl install * Fix tests. * Rename wasi-http -> wasmtime-wasi-http * prtest:full Conditionalize TLS on riscv64gc. * prtest:full Fix formatting, also disable tls on s390x * prtest:full Add a path parameter to wit-bindgen, remove symlink. * prtest:full Fix tests for places where SSL isn't supported. * Update crates/wasi-http/Cargo.toml --------- Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com> Co-authored-by: Pat Hickey <phickey@fastly.com> Co-authored-by: Pat Hickey <pat@moreproductive.org> |
||
 Remo Senekowitsch
|
7eb8914090 |
Chaos mode MVP: Skip branch optimization in MachBuffer (#6039)
* fuzz: Add chaos mode control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: Skip branch optimization with chaos mode Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: Rename chaos engine -> control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * chaos mode: refactoring ControlPlane to be passed through the call stack by reference Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Remo Senekowitsch <contact@remsle.dev> * fuzz: annotate chaos todos Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: cleanup control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: remove control plane from compiler context Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: move control plane into emit state Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: fix remaining compiler errors Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fix tests * refactor emission state ctrl plane accessors Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * centralize conditional compilation of chaos mode Also cleanup a few straggling dependencies on cranelift-control that aren't needed anymore. Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * add cranelift-control to published crates prtest:full Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * add cranelift-control to public crates Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> --------- Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> Co-authored-by: Remo Senekowitsch <contact@remsle.dev> |
||
 Kevin Rizzo
|
3a92aa3d0a |
winch: Initial integration with wasmtime (#6119)
* Adding in trampoline compiling method for ISA * Adding support for indirect call to memory address * Refactoring frame to externalize defined locals, so it removes WASM depedencies in trampoline case * Adding initial version of trampoline for testing * Refactoring trampoline to be re-used by other architectures * Initial wiring for winch with wasmtime * Add a Wasmtime CLI option to select `winch` This is effectively an option to select the `Strategy` enumeration. * Implement `Compiler::compile_function` for Winch Hook this into the `TargetIsa::compile_function` hook as well. Currently this doesn't take into account `Tunables`, but that's left as a TODO for later. * Filling out Winch append_code method * Adding back in changes from previous branch Most of these are a WIP. It's missing trampolines for x64, but a basic one exists for aarch64. It's missing the handling of arguments that exist on the stack. It currently imports `cranelift_wasm::WasmFuncType` since it's what's passed to the `Compiler` trait. It's a bit awkward to use in the `winch_codegen` crate since it mostly operates on `wasmparser` types. I've had to hack in a conversion to get things working. Long term, I'm not sure it's wise to rely on this type but it seems like it's easier on the Cranelift side when creating the stub IR. * Small API changes to make integration easier * Adding in new FuncEnv, only a stub for now * Removing unneeded parts of the old PoC, and refactoring trampoline code * Moving FuncEnv into a separate file * More comments for trampolines * Adding in winch integration tests for first pass * Using new addressing method to fix stack pointer error * Adding test for stack arguments * Only run tests on x86 for now, it's more complete for winch * Add in missing documentation after rebase * Updating based on feedback in draft PR * Fixing formatting on doc comment for argv register * Running formatting * Lock updates, and turning on winch feature flags during tests * Updating configuration with comments to no longer gate Strategy enum * Using the winch-environ FuncEnv, but it required changing the sig * Proper comment formatting * Removing wasmtime-winch from dev-dependencies, adding the winch feature makes this not necessary * Update doc attr to include winch check * Adding winch feature to doc generation, which seems to fix the feature error in CI * Add the `component-model` feature to the cargo doc invocation in CI To match the metadata used by the docs.rs invocation when building docs. * Add a comment clarifying the usage of `component-model` for docs.rs * Correctly order wasmtime-winch and winch-environ in the publish script * Ensure x86 test dependencies are included in cfg(target_arch) * Further constrain Winch tests to x86_64 _and_ unix --------- Co-authored-by: Alex Crichton <alex@alexcrichton.com> Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com> |
||
|
Saúl Cabrera
|
80bfb35072 |
winch: Introduce winch-environ (#6017)
This commit introduces the `winch-environ` crate. This crate's responsibility is to provide a shared implementatation of the `winch_codegen::FuncEnv` trait, which is Winch's function compilation environment, used to resolve module and runtime specific information needed by the code generation, such as resolving all the details about a callee in a WebAssembly module, or resolving specific information from the `VMContext`. As of this change, the implementation only includes the necessary pieces to resolve a function callee in a WebAssembly module. The idea is to evolve the `winch_codegen::FuncEnv` trait as we evolve Winch's code generation. |
||
|
Nick Fitzgerald
|
9ed441e657 |
Introduce the wasmtime-explorer crate (#5975)
This implements Godbolt Compiler Explorer-like functionality for Wasmtime and Cranelift. Given a Wasm module, it compiles the module to native code and then writes a standalone HTML file that gives a split pane view between the WAT and ASM disassemblies. |
||
|
Kevin Rizzo
|
013b35ff32 |
winch: Refactoring wasmtime compiler integration pieces to share more between Cranelift and Winch (#5944)
* Enable the native target by default in winch
Match cranelift-codegen's build script where if no architecture is
explicitly enabled then the host architecture is implicitly enabled.
* Refactor Cranelift's ISA builder to share more with Winch
This commit refactors the `Builder` type to have a type parameter
representing the finished ISA with Cranelift and Winch having their own
typedefs for `Builder` to represent their own builders. The intention is
to use this shared functionality to produce more shared code between the
two codegen backends.
* Moving compiler shared components to a separate crate
* Restore native flag inference in compiler building
This fixes an oversight from the previous commits to use
`cranelift-native` to infer flags for the native host when using default
settings with Wasmtime.
* Move `Compiler::page_size_align` into wasmtime-environ
The `cranelift-codegen` crate doesn't need this and winch wants the same
implementation, so shuffle it around so everyone has access to it.
* Fill out `Compiler::{flags, isa_flags}` for Winch
These are easy enough to plumb through with some shared code for
Wasmtime.
* Plumb the `is_branch_protection_enabled` flag for Winch
Just forwarding an isa-specific setting accessor.
* Moving executable creation to shared compiler crate
* Adding builder back in and removing from shared crate
* Refactoring the shared pieces for the `CompilerBuilder`
I decided to move a couple things around from Alex's initial changes.
Instead of having the shared builder do everything, I went back to
having each compiler have a distinct builder implementation. I
refactored most of the flag setting logic into a single shared location,
so we can still reduce the amount of code duplication.
With them being separate, we don't need to maintain things like
`LinkOpts` which Winch doesn't currently use. We also have an avenue to
error when certain flags are sent to Winch if we don't support them. I'm
hoping this will make things more maintainable as we build out Winch.
I'm still unsure about keeping everything shared in a single crate
(`cranelift_shared`). It's starting to feel like this crate is doing too
much, which makes it difficult to name. There does seem to be a need for
two distinct abstraction: creating the final executable and the handling
of shared/ISA flags when building the compiler. I could make them into
two separate crates, but there doesn't seem to be enough there yet to
justify it.
* Documentation updates, and renaming the finish method
* Adding back in a default temporarily to pass tests, and removing some unused imports
* Fixing winch tests with wrong method name
* Removing unused imports from codegen shared crate
* Apply documentation formatting updates
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
* Adding back in cranelift_native flag inferring
* Adding new shared crate to publish list
* Adding write feature to pass cargo check
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
|
||
 Andrew Brown
|
edfa10d607 |
wasi-threads: an initial implementation (#5484)
This commit includes a set of changes that add initial support for `wasi-threads` to Wasmtime: * feat: remove mutability from the WasiCtx Table This patch adds interior mutability to the WasiCtx Table and the Table elements. Major pain points: * `File` only needs `RwLock<cap_std::fs::File>` to implement `File::set_fdflags()` on Windows, because of [1] * Because `File` needs a `RwLock` and `RwLock*Guard` cannot be hold across an `.await`, The `async` from `async fn num_ready_bytes(&self)` had to be removed * Because `File` needs a `RwLock` and `RwLock*Guard` cannot be dereferenced in `pollable`, the signature of `fn pollable(&self) -> Option<rustix::fd::BorrowedFd>` changed to `fn pollable(&self) -> Option<Arc<dyn AsFd + '_>>` [1] https://github.com/bytecodealliance/system-interface/blob/da238e324e752033f315f09c082ad9ce35d42696/src/fs/fd_flags.rs#L210-L217 * wasi-threads: add an initial implementation This change is a first step toward implementing `wasi-threads` in Wasmtime. We may find that it has some missing pieces, but the core functionality is there: when `wasi::thread_spawn` is called by a running WebAssembly module, a function named `wasi_thread_start` is found in the module's exports and called in a new instance. The shared memory of the original instance is reused in the new instance. This new WASI proposal is in its early stages and details are still being hashed out in the [spec] and [wasi-libc] repositories. Due to its experimental state, the `wasi-threads` functionality is hidden behind both a compile-time and runtime flag: one must build with `--features wasi-threads` but also run the Wasmtime CLI with `--wasm-features threads` and `--wasi-modules experimental-wasi-threads`. One can experiment with `wasi-threads` by running: ```console $ cargo run --features wasi-threads -- \ --wasm-features threads --wasi-modules experimental-wasi-threads \ <a threads-enabled module> ``` Threads-enabled Wasm modules are not yet easy to build. Hopefully this is resolved soon, but in the meantime see the use of `THREAD_MODEL=posix` in the [wasi-libc] repository for some clues on what is necessary. Wiggle complicates things by requiring the Wasm memory to be exported with a certain name and `wasi-threads` also expects that memory to be imported; this build-time obstacle can be overcome with the `--import-memory --export-memory` flags only available in the latest Clang tree. Due to all of this, the included tests are written directly in WAT--run these with: ```console $ cargo test --features wasi-threads -p wasmtime-cli -- cli_tests ``` [spec]: https://github.com/WebAssembly/wasi-threads [wasi-libc]: https://github.com/WebAssembly/wasi-libc This change does not protect the WASI implementations themselves from concurrent access. This is already complete in previous commits or left for future commits in certain cases (e.g., wasi-nn). * wasi-threads: factor out process exit logic As is being discussed [elsewhere], either calling `proc_exit` or trapping in any thread should halt execution of all threads. The Wasmtime CLI already has logic for adapting a WebAssembly error code to a code expected in each OS. This change factors out this logic to a new function, `maybe_exit_on_error`, for use within the `wasi-threads` implementation. This will work reasonably well for CLI users of Wasmtime + `wasi-threads`, but embedders will want something better in the future: when a `wasi-threads` threads fails, they may not want their application to exit. Handling this is tricky, because it will require cancelling the threads spawned by the `wasi-threads` implementation, something that is not trivial to do in Rust. With this change, we defer that work until later in order to provide a working implementation of `wasi-threads` for experimentation. [elsewhere]: https://github.com/WebAssembly/wasi-threads/pull/17 * review: work around `fd_fdstat_set_flags` In order to make progress with wasi-threads, this change temporarily works around limitations induced by `wasi-common`'s `fd_fdstat_set_flags` to allow `&mut self` use in the implementation. Eventual resolution is tracked in https://github.com/bytecodealliance/wasmtime/issues/5643. This change makes several related helper functions (e.g., `set_fdflags`) take `&mut self` as well. * test: use `wait`/`notify` to improve `threads.wat` test Previously, the test simply executed in a loop for some hardcoded number of iterations. This changes uses `wait` and `notify` and atomic operations to keep track of when the spawned threads are done and join on the main thread appropriately. * various fixes and tweaks due to the PR review --------- Signed-off-by: Harald Hoyer <harald@profian.com> Co-authored-by: Harald Hoyer <harald@profian.com> Co-authored-by: Alex Crichton <alex@alexcrichton.com> |
||
 Thibault Charbonnier
|
e835255fbf |
c-api: add Wasmtime version macros to wasmtime.h (#5651)
* Add several `WASMTIME_VERSION_*` macros to `wasmtime.h`.
* Update `scripts/publish.rs`
* To set these macros as per the new version in `./Cargo.toml` during
`./publish bump`.
* To verify the macros match the version in `./Cargo.toml` during
`./publish verify`.
Fix #5635
|
||
 Jamey Sharp
|
915801551b |
Delete old cranelift-preopt crate (#5642)
Most of these optimizations are in the egraph `cprop.isle` rules now, making a separate crate unnecessary. Also I think the `udiv` optimizations here are straight-up wrong (doing signed instead of unsigned division, and panicking instead of preserving traps on division by zero) so I'm guessing this crate isn't seriously used anywhere. At the least, bjorn3 confirms that cg_clif doesn't use this, and I've verified that Wasmtime doesn't either. Closes #1090. |
||
|
Alex Crichton
|
2329ecc341 |
Add a wasmtime::component::bindgen! macro (#5317)
* Import Wasmtime support from the `wit-bindgen` repo This commit imports the `wit-bindgen-gen-host-wasmtime-rust` crate from the `wit-bindgen` repository into the upstream Wasmtime repository. I've chosen to not import the full history here since the crate is relatively small and doesn't have a ton of complexity. While the history of the crate is quite long the current iteration of the crate's history is relatively short so there's not a ton of import there anyway. The thinking is that this can now continue to evolve in-tree. * Refactor `wasmtime-component-macro` a bit Make room for a `wit_bindgen` macro to slot in. * Add initial support for a `bindgen` macro * Add tests for `wasmtime::component::bindgen!` * Improve error forgetting `async` feature * Add end-to-end tests for bindgen * Add an audit of `unicase` * Add a license to the test-helpers crate * Add vet entry for `pulldown-cmark` * Update publish script with new crate * Try to fix publish script * Update audits * Update lock file |
||
|
Saúl Cabrera
|
835abbcd11 |
Initial skeleton for Winch (#4907)
* Initial skeleton for Winch
This commit introduces the initial skeleton for Winch, the "baseline"
compiler.
This skeleton contains mostly setup code for the ISA, ABI, registers,
and compilation environment abstractions. It also includes the
calculation of function local slots.
As of this commit, the structure of these abstractions looks like the
following:
+------------------------+
| v
+----------+ +-----+ +-----------+-----+-----------------+
| Compiler | --> | ISA | --> | Registers | ABI | Compilation Env |
+----------+ +-----+ +-----------+-----+-----------------+
| ^
+------------------------------+
* Compilation environment will hold a reference to the function data
* Add basic documentation to the ABI trait
* Enable x86 and arm64 in cranelift-codegen
* Add reg_name function for x64
* Introduce the concept of a MacroAssembler and Assembler
This commit introduces the concept of a MacroAsesembler and
Assembler. The MacroAssembler trait will provide a high enough
interface across architectures so that each ISA implementation can use their own low-level
Assembler implementation to fulfill the interface. Each Assembler will
provide a 1-1 mapping to each ISA instruction.
As of this commit, only a partial debug implementation is provided for
the x64 Assembler.
* Add a newtype over PReg
Adds a newtype `Reg` over regalloc2::PReg; this ensures that Winch
will operate only on the concept of `Reg`. This change is temporary
until we have the necessary machinery to share a common Reg
abstraction via `cranelift_asm`
* Improvements to local calcuation
- Add `LocalSlot::addressed_from_sp`
- Use `u32` for local slot and local sizes calculation
* Add helper methods to ABIArg
Adds helper methods to retrieve register and type information from the argument
* Make locals_size public in frame
* Improve x64 register naming depending on size
* Add new methods to the masm interface
This commit introduces the ability for the MacroAssembler to reserve
stack space, get the address of a given local and perform a stack
store based on the concept of `Operand`s.
There are several motivating factors to introduce the concept of an
Operand:
- Make the translation between Winch and Cranelift easier;
- Make dispatching from the MacroAssembler to the underlying Assembler
- easier by minimizing the amount of functions that we need to define
- in order to satisfy the store/load combinations
This commit also introduces the concept of a memory address, which
essentially describes the addressing modes; as of this commit only one
addressing mode is supported. We'll also need to verify that this
structure will play nicely with arm64.
* Blank masm implementation for arm64
* Implementation of reserve_stack, local_address, store and fp_offset
for x64
* Implement function prologue and argument register spilling
* Add structopt and wat
* Fix debug instruction formatting
* Make TargetISA trait publicly accessible
* Modify the MacroAssembler finalize siganture to return a slice of strings
* Introduce a simple CLI for Winch
To be able to compile Wasm programs with Winch independently. Mostly
meant for testing / debugging
* Fix bug in x64 assembler mov_rm
* Remove unused import
* Move the stack slot calculation to the Frame
This commit moves the calculation of the stack slots to the frame
handler abstraction and also includes the calculation of the limits
for the function defined locals, which will be used to zero the locals
that are not associated to function arguments
* Add i32 and i64 constructors to local slots
* Introduce the concept of DefinedLocalsRange
This commit introduces `DefinedLocalsRange` to track the stack offset
at which the function-defined locals start and end; this is later used
to zero-out that stack region
* Add constructors for int and float registers
* Add a placeholder stack implementation
* Add a regset abstraction to track register availability
Adds a bit set abstraction to track register availability for register
allocation.
The bit set has no specific knowledge about physical registers, it
works on the register's hardware encoding as the source of truth.
Each RegSet is expected to be created with the universe of allocatable
registers per ISA when starting the compilation of a particular function.
* Add an abstraction over register and immediate
This is meant to be used as the source for stores.
* Add a way to zero local slots and an initial skeletion of regalloc
This commit introduces `zero_local_slots` to the MacroAssembler; which
ensures that function defined locals are zeroed out when starting the
function body.
The algorithm divides the defined function locals stack range
into 8 byte slots and stores a zero at each address. This process
relies on register allocation if the amount of slots that need to be
initialized is greater than 1. In such case, the next available
register is requested to the register set and it's used to store a 0,
which is then stored at every local slot
* Update to wasmparser 0.92
* Correctly track if the regset has registers available
* Add a result entry to the ABI signature
This commuit introduces ABIResult as part of the ABISignature;
this struct will track how function results are stored; initially it
will consiste of a single register that will be requested to the
register allocator at the end of the function; potentially causing a spill
* Move zero local slots and add more granular methods to the masm
This commit removes zeroing local slots from the MacroAssembler and
instead adds more granular methods to it (e.g `zero`, `add`).
This allows for better code sharing since most of the work done by the
algorithm for zeroing slots will be the same in all targets, except
for the binary emissions pieces, which is what gets delegated to the masm
* Use wasmparser's visitor API and add initial support for const and add
This commit adds initial support for the I32Const and I32
instructions; this involves adding a minimum for register
allocation. Note that some regalloc pieces are still incomplete, since
for the current set of supported instructions they are not needed.
* Make the ty field public in Local
* Add scratch_reg to the abi
* Add a method to get a particular local from the Frame
* Split the compilation environment abstraction
This commit splits the compilation environment into two more concise
abstractions:
1. CodeGen: the main abstraction for code generation
2. CodeGenContext: abstraction that shares the common pieces for
compilation; these pieces are shared between the code generator and
the register allocator
* Add `push` and `load` to the MacroAssembler
* Remove dead code warnings for unused paths
* Map ISA features to cranelift-codegen ISA features
* Apply formatting
* Fix Cargo.toml after a bad rebase
* Add component-compiler feature
* Use clap instead of structopt
* Add winch to publish.rs script
* Minor formatting
* Add tests to RegSet and fix two bugs when freeing and checking for
register availability
* Add tests to Stack
* Free source register after a non-constant i32 add
* Improve comments
- Remove unneeded comments
- And improve some of the TODO items
* Update default features
* Drop the ABI generic param and pass the word_size information directly
To avoid dealing with dead code warnings this commit passes the word
size information directly, since it's the only piece of information
needed from the ABI by Codegen until now
* Remove dead code
This piece of code will be put back once we start integrating Winch
with Wasmtime
* Remove unused enum variant
This variant doesn't get constructed; it should be added back once a
backend is added and not enabled by default or when Winch gets
integrated into Wasmtime
* Fix unused code in regset tests
* Update spec testsuite
* Switch the visitor pattern for a simpler operator match
This commit removes the usage of wasmparser's visitor pattern and
instead defaults to a simpler operator matching approach. This removes
the complexity of having to define all the visitor trait functions at once.
* Use wasmparser's Visitor trait with a different macro strategy
This commit puts back wasmparser's Visitor trait, with a sigle;
simpler macro, only used for unsupported operators.
* Restructure Winch
This commit restuructures Winch's parts. It divides the initial
approach into three main crates: `winch-codegen`,`wasmtime-winch` and `winch-tools`.
`wasmtime-winch` is reponsible for the Wasmtime-Winch integration.
`winch-codegen` is solely responsible for code generation.
`winch-tools` is CLI tool to compile Wasm programs, mainly for testing purposes.
* Refactor zero local slots
This commit moves the logic of zeroing local slots from the codegen
module into a method with a default implementation in the
MacroAssembler trait: `zero_mem_range`.
The refactored implementation is very similar to the previous
implementation with the only difference
that it doesn't allocates a general-purpose register; it instead uses
the register allocator to retrieve the scratch register and uses this
register to unroll the series of zero stores.
* Tie the codegen creation to the ISA ABI
This commit makes the relationship between the ISA ABI and the codegen
explicit. This allows us to pass down ABI-specific bit and pieces to
the codegeneration. In this case the only concrete piece that we need
is the ABI word size.
* Mark winch as publishable directory
* Revamp winch docs
This commit ensures that all the code comments in Winch are compliant
with the syle used in the rest of Wasmtime's codebase.
It also imptoves, generally the quality of the comments in some modules.
* Panic when using multi-value when the target is aarch64
Similar to x64, this commit ensures that the abi signature of the
current function doesn't use multi-value returns
* Document the usage of directives
* Use endianness instead of endianess in the ISA trait
* Introduce a three-argument form in the MacroAssembler
This commit introduces the usage of three-argument form for the
MacroAssembler interface. This allows for a natural mapping for
architectures like aarch64. In the case of x64, the implementation can
simply restrict the implementation asserting for equality in two of
the arguments of defaulting to a differnt set of instructions.
As of this commit, the implementation of `add` panics if the
destination and the first source arguments are not equal; internally
the x64 assembler implementation will ensure that all the allowed
combinations of `add` are satisfied. The reason for panicking and not
emitting a `mov` followed by an `add` for example is simply because register
allocation happens right before calling `add`, which ensures any
register-to-register moves, if needed.
This implementation will evolve in the future and this panic will be
lifted if needed.
* Improve the documentation for the MacroAssembler.
Documents the usage of three-arg form and the intention around the
high-level interface.
* Format comments in remaining modules
* Clean up Cargo.toml for winch pieces
This commit adds missing fields to each of Winch's Cargo.toml.
* Use `ModuleTranslation::get_types()` to derive the function type
* Assert that start range is always word-size aligned
|
||
|
Alex Crichton
|
204d4c332c |
Increase attempts and timeout in publish script (#5084)
The 2.0.0 publication failed due to the index not being updated, so do more attempts and wait longer to try to get things working. |
||
 Afonso Bordado
|
4639e85c4e |
Flush Icache on AArch64 Windows (#4997)
* cranelift: Add FlushInstructionCache for AArch64 on Windows This was previously done on #3426 for linux. * wasmtime: Add FlushInstructionCache for AArch64 on Windows This was previously done on #3426 for linux. * cranelift: Add MemoryUse flag to JIT Memory Manager This allows us to keep the icache flushing code self-contained and not leak implementation details. This also changes the windows icache flushing code to only flush pages that were previously unflushed. * Add jit-icache-coherence crate * cranelift: Use `jit-icache-coherence` * wasmtime: Use `jit-icache-coherence` * jit-icache-coherence: Make rustix feature additive Mutually exclusive features cause issues. * wasmtime: Remove rustix from wasmtime-jit We now use it via jit-icache-coherence * Rename wasmtime-jit-icache-coherency crate * Use cfg-if in wasmtime-jit-icache-coherency crate * Use inline instead of inline(always) * Add unsafe marker to clear_cache * Conditionally compile all rustix operations membarrier does not exist on MacOS * Publish `wasmtime-jit-icache-coherence` * Remove explicit windows check This is implied by the target_os = "windows" above * cranelift: Remove len != 0 check This is redundant as it is done in non_protected_allocations_iter * Comment cleanups Thanks @akirilov-arm! * Make clear_cache safe * Rename pipeline_flush to pipeline_flush_mt * Revert "Make clear_cache safe" This reverts commit 21165d81c9030ed9b291a1021a367214d2942c90. * More docs! * Fix pipeline_flush reference on clear_cache * Update more docs! * Move pipeline flush after `mprotect` calls Technically the `clear_cache` operation is a lie in AArch64, so move the pipeline flush after the `mprotect` calls so that it benefits from the implicit cache cleaning done by it. * wasmtime: Remove rustix backend from icache crate * wasmtime: Use libc for macos * wasmtime: Flush icache on all arch's for windows * wasmtime: Add flags to membarrier call |
||
 Chris Fallin
|
2be12a5167 |
egraph-based midend: draw the rest of the owl (productionized). (#4953)
* egraph-based midend: draw the rest of the owl. * Rename `egg` submodule of cranelift-codegen to `egraph`. * Apply some feedback from @jsharp during code walkthrough. * Remove recursion from find_best_node by doing a single pass. Rather than recursively computing the lowest-cost node for a given eclass and memoizing the answer at each eclass node, we can do a single forward pass; because every eclass node refers only to earlier nodes, this is sufficient. The behavior may slightly differ from the earlier behavior because we cannot short-circuit costs to zero once a node is elaborated; but in practice this should not matter. * Make elaboration non-recursive. Use an explicit stack instead (with `ElabStackEntry` entries, alongside a result stack). * Make elaboration traversal of the domtree non-recursive/stack-safe. * Work analysis logic in Cranelift-side egraph glue into a general analysis framework in cranelift-egraph. * Apply static recursion limit to rule application. * Fix aarch64 wrt dynamic-vector support -- broken rebase. * Topo-sort cranelift-egraph before cranelift-codegen in publish script, like the comment instructs me to! * Fix multi-result call testcase. * Include `cranelift-egraph` in `PUBLISHED_CRATES`. * Fix atomic_rmw: not really a load. * Remove now-unnecessary PartialOrd/Ord derivations. * Address some code-review comments. * Review feedback. * Review feedback. * No overlap in mid-end rules, because we are defining a multi-constructor. * rustfmt * Review feedback. * Review feedback. * Review feedback. * Review feedback. * Remove redundant `mut`. * Add comment noting what rules can do. * Review feedback. * Clarify comment wording. * Update `has_memory_fence_semantics`. * Apply @jameysharp's improved loop-level computation. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix suggestion commit. * Fix off-by-one in new loop-nest analysis. * Review feedback. * Review feedback. * Review feedback. * Use `Default`, not `std::default::Default`, as per @fitzgen Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com> * Apply @fitzgen's comment elaboration to a doc-comment. Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com> * Add stat for hitting the rewrite-depth limit. * Some code motion in split prelude to make the diff a little clearer wrt `main`. * Take @jameysharp's suggested `try_into()` usage for blockparam indices. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Take @jameysharp's suggestion to avoid double-match on load op. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix suggestion (add import). * Review feedback. * Fix stack_load handling. * Remove redundant can_store case. * Take @jameysharp's suggested improvement to FuncEGraph::build() logic Co-authored-by: Jamey Sharp <jamey@minilop.net> * Tweaks to FuncEGraph::build() on top of suggestion. * Take @jameysharp's suggested clarified condition Co-authored-by: Jamey Sharp <jamey@minilop.net> * Clean up after suggestion (unused variable). * Fix loop analysis. * loop level asserts * Revert constant-space loop analysis -- edge cases were incorrect, so let's go with the simple thing for now. * Take @jameysharp's suggestion re: result_tys Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix up after suggestion * Take @jameysharp's suggestion to use fold rather than reduce Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fixup after suggestion * Take @jameysharp's suggestion to remove elaborate_eclass_use's return value. * Clarifying comment in terminator insts. Co-authored-by: Jamey Sharp <jamey@minilop.net> Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com> |
||
|
Alex Crichton
|
7b311004b5 |
Leverage Cargo's workspace inheritance feature (#4905)
* Leverage Cargo's workspace inheritance feature This commit is an attempt to reduce the complexity of the Cargo manifests in this repository with Cargo's workspace-inheritance feature becoming stable in Rust 1.64.0. This feature allows specifying fields in the root workspace `Cargo.toml` which are then reused throughout the workspace. For example this PR shares definitions such as: * All of the Wasmtime-family of crates now use `version.workspace = true` to have a single location which defines the version number. * All crates use `edition.workspace = true` to have one default edition for the entire workspace. * Common dependencies are listed in `[workspace.dependencies]` to avoid typing the same version number in a lot of different places (e.g. the `wasmparser = "0.89.0"` is now in just one spot. Currently the workspace-inheritance feature doesn't allow having two different versions to inherit, so all of the Cranelift-family of crates still manually specify their version. The inter-crate dependencies, however, are shared amongst the root workspace. This feature can be seen as a method of "preprocessing" of sorts for Cargo manifests. This will help us develop Wasmtime but shouldn't have any actual impact on the published artifacts -- everything's dependency lists are still the same. * Fix wasi-crypto tests |