* Rust: add option for path to wit-bindgen runtime.
This change adds an optional path to the `wit-bindgen` runtime to use for Rust
code generation.
This will allow a different crate from `wit-bindgen` to generate bindings while
also re-exporting `wit-bindgen::rt` at a known path for the generated code to
use.
* Rust: add `runtime_path` macro option and a test.
* C: Namespace types inside of annoymous types
* Go generator should use c generator's name generation
* Optionaly namespace inner types for all anonymous types
* Test more
* provide more control over type ownership
This is the guest version of https://github.com/bytecodealliance/wasmtime/pull/6648.
This replaces the duplicate_if_necessary parameter to the Rust binding generator
with a new ownership parameter which provides finer-grained control over whether
and how generated types own their fields.
The default is `Ownership::Owning`, which means types own their fields
regardless of how they are used in functions. These types are passed by
reference when used as parameters to guest-exported functions. Note that this
also affects how unnamed types (e.g. `list<list<string>>`) are passed: using a
reference only at the top level (e.g. `&[Vec<String>]` instead of `&[&[&str]]`,
which is more difficult to construct when using non-'static data).
The other option is `Ownership::Borrowing`, which includes a
`duplicate_if_necessary` field, providing the same code generation strategy as
was used prior to this change.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* fix tests
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* fix tests, part 2
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Update Rust codegen for new WIT changes
* Update the markdown generation for new WIT
* more-rust
* Update the C generator for new WIT
* Get Rust guest tests compiling
* Get rust host tests working
* Get all C tests passing
* Turn off teavm-java and Go tests for now
* Fixup lockfile after rebase
* Bump crate versions
* Update dependency sources
* Update to wasmtime dep
* Update how types are generated in C
Prior to this commit each interface used in C would generate a separate
type per import and per export. For example if a `world` both imported
and exported an interface then distinct, but defined the same way, types
would be generated. Furthermore this same logic was extended to
types-in-worlds which ended up not working well and causing #544.
This commit overhauls how types are generated in C. All used interfaces
now generate one set of types, regardless of whether the interface is
used in an import or an export. Additionally this enables fixing #544 by
generating types for worlds only once at the end instead of
incrementally throughout.
Along the way this updates many names of generated types to be more
appropriate to their scoping constraints. For example types in
interfaces are now namespaced by the name of the interface. Functions
continue to be namespaced by the name of the import/export and disregard
the name of the interface. Ambient types like `record<T, E>` continue to
be prefixed by the name of the world that bindings are generated for.
Closes#544
* Disable go runtime tests
* Disable go codegen tests temporarily
Previously when a single WIT type was used in a "borrowed" position, or
a parameter to an import, and an "owned" position then the type would
have two copies of its definition generated if it internally contained a
list. This was intended to signify that for import parameters ownership
wasn't necessary so borrowed values could be passed in.
In practice though I don't think this was the right default to have.
This surprisingly generates two types when one might expect one and
otherwise if a type if received from an export and expected to be
passed to an import then it's not easy to do so since it must be
converted to the borrowed type.
This commit moves the preexisting behavior of generating duplicate types
behind a new `duplicate_if_necessary` option. The default behavior is to
no longer generate two types, but instead just one. Imports now generate
their types with a `&` in front if it's otherwise owned internally to
signify that ownership isn't required.
Closes#535
Per #495, Maven has proven quite unreliable for the past few weeks, so
now we download and execute the relevant TeaVM jars directly, which
should be more reliable.
This also fixes the lift-lower-foreign.wit codegen test for Java.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
This pulls in type analysis code from Wasmtime which has a more modern
notion of param/result as owned/borrowed and additionally removes the
need for a `default_param_mode` which caused #513 in the first place.
Closes#513
* created wit-bindgen-gen-guest-go crate and added to the clap CLI
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* rewrote import and export using a more generic conversion method
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: records working
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* simply code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: tuple works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: list works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: string works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: option works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* implemented result type
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: result works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: result option work
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added codegen for go bindgen. Further work needs to pass all the tests
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* get more tests passed, including conventions
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* now flags and alias are working
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: implemented multi-return
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
cargo clippy
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: implemented union, enum and variantss
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* rewrite lower's Result for imports
* avoid Go keywords conflict with wit's variable names
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: all codegen tests passed.
The issue with "fragment is larger than or outside of variable" is
a known issue and has been resolved in tinygo v0.26.
However, tinygo v0.26 introduces a regression for wasm/wasi target
where the compiler panics when size 0 passed to malloc. See
https://github.com/tinygo-org/tinygo/pull/3303 for details
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* refactor: add gen-guest-c dependency to gen-guest-go.
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* cargo lock file
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: passed new tests associated cross-interface feature
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* formatted code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* gh action: installed tinygo
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* install go 1.19
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* generate C files together with Go files
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added option-result test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fix a typo in wit-bindgen-cli doc
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* updated README to include gen-guest-go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* removed option-result.wit
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fix the broken CLI
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added go to runtime test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* free C parameters and variables
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* removed memory free for return variables in export and invoke arguments in import
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* implemented lists
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed a small bug around lowering nested option types
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* improve readability of code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* changed the variant's package name to the world's name.
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: re-implement results
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Fixed a small reference issue for lowering result
* re-implement variants and unions
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed a few small bugs around variants and union types. Added variant runtime tests
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* turn off gc-leaking in tinygo 0.26
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* simplify code and add multi-return to lowering
* Added runtime tests for list, number and variant
* Added flavorful runtime test
* Added tests for many_aguments, records and smoke
* Added strings and unions runtime tests
* removed http from runtime test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added go bindgen build step to ci
* updated tinygo to 0.27. All tests should pass now
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed go version to 1.20 instead of 1.2
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* disable string test because unicode hasn't been handled properly
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* deleted wasm abi flag
* merged main
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Move go's files to it's own subfolder
* move C gen files to its own folder
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* update go from 1.19 to 1.20
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* changed tinygo crate version to 0.1.0
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* diable variants test for go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Fixed a missing argument to encode
* Fix referring to foreign imported paths in go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* disable lift-foreign tests for now
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* run rustfmt
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
---------
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Rename `wit-bindgen-gen-guest-rust` to `wit-bindgen`
This commit creates a crate called `wit-bindgen` to be used from Rust.
While the name `wit-bindgen-gen-guest-rust` makes sense in the context
of other generators when using it specifically from Rust I think it
makes more sense to have the crate simply called `wit-bindgen` for
integration since the only context that name is used within is guest
bindings generator for Rust code.
* Add release CI automation for `wit-bindgen`
This is cobbled together from the `wasm-tools` and `wasmtime`
repositories and the idea is:
* A `publish.rs` script is added for managing versions and publication
to crates.io.
* Tags will upload binaries to GitHub releases
* CI is added for binary release builds of the CLI
* Fix copy/paste error
Per #495, Maven has started failing to resolve dependencies on Windows
in the GitHub action, although it works fine on Mac, Linux, and also
(locally) on Windows. For the time being, I'm disabling these tests
so the failures are not a distraction while we determine a proper
solution.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* gen-guest-c: Fix cabi_realloc behavior when new_size is zero
Match the behavior of the Rust guest cabi_realloc by returning `align`
if new_size == 0.
* guest-rust: Add an assert to cabi_realloc
The code generator is expected not to emit calls that might have
`old_size == 0 && new_size == 0`, but if that condition is not met
the resulting call to `alloc::realloc` is documented as having undefined
behavior.
* Add test exercising cabi_realloc new_size = 0
* Refactor root `Cargo.toml` a bit
* Update some wasm-tools dependencies
* Use a `[patch]` for Wasmtime to reflect how this will eventually be a
crates.io dependency
* Use `workspace = true` to avoid duplicating path names.
* Remove an unneeded futures-util dep
* Remove the JS host generator
This commit removes the JS host generator from this repository as it now
lives at bytecodealliance/js-component-tools. The runtime tests will be
updated in a subsequent commit to use Wasmtime as a host for executing
components.
This additionally removes the demo as it's no longer possible to build
it in just this repository. The demo has slowly been getting less useful
over time as generators are removed from this repository. Additionally
it's always been a demo and not much more so it's more-or-less served
its purpose and doesn't seem to have a great place to fit any more. This
can possibly exist out-of-tree eventually in a separate location that
pulls together all known generators, but until that time it seems best
to just remove it for now.
Closes#395
* Update the CLI with just guest generators
No need for a `guest` subcommand any more since all supported generators
are just guest generators.
* Let tests run for the `wit-bindgen` binary
This'll make sure that it builds as part of `cargo test`
* Add scaffolding to run tests in Wasmtime
Tests will unfortunately take longer due to wasmtime build times being
somewhat nontrivial and additionally rebuilding the java components on
each invocation is costly, but for now this should keep the tests
relatively simple and easy to maintain by having all the logic be in one
place rather than across multiple crates.
* Migrate flavorful test
* Migrate the lists test
* Migrate the many arguments test
* Migrate the records test
* Migrate the smoke test
* Bring back the unions host test
* Migrate the variants test
* Migrate strings test to Wasmtime
* Remove invalid runtime test
This is a test for host generators which is no longer being tested in
this repository.
* Remove exports_only test
This is a JS host test primarily
* Fix a rustfmt error
* Update the Rust guest generator
This commit updates the `wit-parser` and `wit-component` dependencies to
the latest 0.4.0 versions on crates.io, namely pulling in support for
`use`. This updates the `*.wit` codegen tests and gets all the codegen
tests working for the `gen-guest-rust` crate. This does not get the
entire repository's tests running.
The major changes here are:
* The `WorldGenerator` trait has been adapted for worlds and the new
APIs of `wit-parser`.
* The Rust guest generator now supports bare function imports as well as
bare function exports.
* The syntax in the Rust macro now mirrors the syntax in the Wasmtime
`bindgen!` macro where `generate!("foo")` will look for the document
`foo.wit` in `$CARGO_MANIFEST_DIR/wit/foo.wit` and look for a `default
world` inside there.
* The codegen and runtime macros have been refactored to get separated
and additionally have conditional compilation working so one generator
can be tested in isolation without requiring all others working.
* The `rust-macro-shared` crate is now removed since Wasmtime no longer
resides here and it's merged directly into the `guest-rust-macro`
crate. Note that many changes happened here since the macro syntax and
integration with `wit-parser` has changed.
Note that no new tests were added in this commit, and that'll be a
follow-up.
* Add support for using types across interfaces to Rust
* Support renaming interfaces in imports
* Get all codegen tests for JS passing again
* Fix build of wasi_snapshot_preview1 for this repo
* Get unions tests building again
* Get the flavorful test compiling again
* Get the records test compiling again
* Get the many arguments test compiling again
* Get the smoke test compiling again
* Get numbers test compiling again
* Get list test compiling again
* Get variants test compiling again
* Get results test compiling again
* Get invalid test compiling again
* Get all runtime JS tests passing
Change a few bits and pieces of the JS host generator as necessary and
temporarily disable other guest generators.
* Use git repo for wit-{component,parser} patch
* Update C generator for new WIT
* Get the bare bones of teavm-java working
Runtime tests will need some more work to get passing.
* Get CLI and markdown crate building again
* Update build of demo
* Use updated crates.io dep for wit-component
* fix Java runtime test regressions (#1)
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Update wasm-tools on CI
* Review comments
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Co-authored-by: Joel Dice <joel.dice@fermyon.com>
