Per #495, Maven has proven quite unreliable for the past few weeks, so
now we download and execute the relevant TeaVM jars directly, which
should be more reliable.
This also fixes the lift-lower-foreign.wit codegen test for Java.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* created wit-bindgen-gen-guest-go crate and added to the clap CLI
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* rewrote import and export using a more generic conversion method
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: records working
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* simply code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: tuple works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: list works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: string works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: option works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* implemented result type
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: result works
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: result option work
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added codegen for go bindgen. Further work needs to pass all the tests
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* get more tests passed, including conventions
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* now flags and alias are working
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: implemented multi-return
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
cargo clippy
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: implemented union, enum and variantss
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* rewrite lower's Result for imports
* avoid Go keywords conflict with wit's variable names
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: all codegen tests passed.
The issue with "fragment is larger than or outside of variable" is
a known issue and has been resolved in tinygo v0.26.
However, tinygo v0.26 introduces a regression for wasm/wasi target
where the compiler panics when size 0 passed to malloc. See
https://github.com/tinygo-org/tinygo/pull/3303 for details
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* refactor: add gen-guest-c dependency to gen-guest-go.
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* cargo lock file
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: passed new tests associated cross-interface feature
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* formatted code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* gh action: installed tinygo
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* install go 1.19
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* generate C files together with Go files
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added option-result test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fix a typo in wit-bindgen-cli doc
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* updated README to include gen-guest-go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* removed option-result.wit
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fix the broken CLI
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added go to runtime test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* free C parameters and variables
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* removed memory free for return variables in export and invoke arguments in import
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* implemented lists
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed a small bug around lowering nested option types
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* improve readability of code
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* changed the variant's package name to the world's name.
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* milestone: re-implement results
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Fixed a small reference issue for lowering result
* re-implement variants and unions
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed a few small bugs around variants and union types. Added variant runtime tests
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* turn off gc-leaking in tinygo 0.26
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* simplify code and add multi-return to lowering
* Added runtime tests for list, number and variant
* Added flavorful runtime test
* Added tests for many_aguments, records and smoke
* Added strings and unions runtime tests
* removed http from runtime test
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* added go bindgen build step to ci
* updated tinygo to 0.27. All tests should pass now
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* fixed go version to 1.20 instead of 1.2
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* disable string test because unicode hasn't been handled properly
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* deleted wasm abi flag
* merged main
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Move go's files to it's own subfolder
* move C gen files to its own folder
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* update go from 1.19 to 1.20
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* changed tinygo crate version to 0.1.0
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* diable variants test for go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Fixed a missing argument to encode
* Fix referring to foreign imported paths in go
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* disable lift-foreign tests for now
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* run rustfmt
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
---------
Signed-off-by: Jiaxiao Zhou <jiazho@microsoft.com>
* Rename `wit-bindgen-gen-guest-rust` to `wit-bindgen`
This commit creates a crate called `wit-bindgen` to be used from Rust.
While the name `wit-bindgen-gen-guest-rust` makes sense in the context
of other generators when using it specifically from Rust I think it
makes more sense to have the crate simply called `wit-bindgen` for
integration since the only context that name is used within is guest
bindings generator for Rust code.
* Add release CI automation for `wit-bindgen`
This is cobbled together from the `wasm-tools` and `wasmtime`
repositories and the idea is:
* A `publish.rs` script is added for managing versions and publication
to crates.io.
* Tags will upload binaries to GitHub releases
* CI is added for binary release builds of the CLI
* Fix copy/paste error
* Refactor root `Cargo.toml` a bit
* Update some wasm-tools dependencies
* Use a `[patch]` for Wasmtime to reflect how this will eventually be a
crates.io dependency
* Use `workspace = true` to avoid duplicating path names.
* Remove an unneeded futures-util dep
* Remove the JS host generator
This commit removes the JS host generator from this repository as it now
lives at bytecodealliance/js-component-tools. The runtime tests will be
updated in a subsequent commit to use Wasmtime as a host for executing
components.
This additionally removes the demo as it's no longer possible to build
it in just this repository. The demo has slowly been getting less useful
over time as generators are removed from this repository. Additionally
it's always been a demo and not much more so it's more-or-less served
its purpose and doesn't seem to have a great place to fit any more. This
can possibly exist out-of-tree eventually in a separate location that
pulls together all known generators, but until that time it seems best
to just remove it for now.
Closes#395
* Update the CLI with just guest generators
No need for a `guest` subcommand any more since all supported generators
are just guest generators.
* Let tests run for the `wit-bindgen` binary
This'll make sure that it builds as part of `cargo test`
* Add scaffolding to run tests in Wasmtime
Tests will unfortunately take longer due to wasmtime build times being
somewhat nontrivial and additionally rebuilding the java components on
each invocation is costly, but for now this should keep the tests
relatively simple and easy to maintain by having all the logic be in one
place rather than across multiple crates.
* Migrate flavorful test
* Migrate the lists test
* Migrate the many arguments test
* Migrate the records test
* Migrate the smoke test
* Bring back the unions host test
* Migrate the variants test
* Migrate strings test to Wasmtime
* Remove invalid runtime test
This is a test for host generators which is no longer being tested in
this repository.
* Remove exports_only test
This is a JS host test primarily
* Fix a rustfmt error
* Update the Rust guest generator
This commit updates the `wit-parser` and `wit-component` dependencies to
the latest 0.4.0 versions on crates.io, namely pulling in support for
`use`. This updates the `*.wit` codegen tests and gets all the codegen
tests working for the `gen-guest-rust` crate. This does not get the
entire repository's tests running.
The major changes here are:
* The `WorldGenerator` trait has been adapted for worlds and the new
APIs of `wit-parser`.
* The Rust guest generator now supports bare function imports as well as
bare function exports.
* The syntax in the Rust macro now mirrors the syntax in the Wasmtime
`bindgen!` macro where `generate!("foo")` will look for the document
`foo.wit` in `$CARGO_MANIFEST_DIR/wit/foo.wit` and look for a `default
world` inside there.
* The codegen and runtime macros have been refactored to get separated
and additionally have conditional compilation working so one generator
can be tested in isolation without requiring all others working.
* The `rust-macro-shared` crate is now removed since Wasmtime no longer
resides here and it's merged directly into the `guest-rust-macro`
crate. Note that many changes happened here since the macro syntax and
integration with `wit-parser` has changed.
Note that no new tests were added in this commit, and that'll be a
follow-up.
* Add support for using types across interfaces to Rust
* Support renaming interfaces in imports
* Get all codegen tests for JS passing again
* Fix build of wasi_snapshot_preview1 for this repo
* Get unions tests building again
* Get the flavorful test compiling again
* Get the records test compiling again
* Get the many arguments test compiling again
* Get the smoke test compiling again
* Get numbers test compiling again
* Get list test compiling again
* Get variants test compiling again
* Get results test compiling again
* Get invalid test compiling again
* Get all runtime JS tests passing
Change a few bits and pieces of the JS host generator as necessary and
temporarily disable other guest generators.
* Use git repo for wit-{component,parser} patch
* Update C generator for new WIT
* Get the bare bones of teavm-java working
Runtime tests will need some more work to get passing.
* Get CLI and markdown crate building again
* Update build of demo
* Use updated crates.io dep for wit-component
* fix Java runtime test regressions (#1)
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Update wasm-tools on CI
* Review comments
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Co-authored-by: Joel Dice <joel.dice@fermyon.com>
* Bump version of dependency crates
This commit updates this repository to published versions of the
`wasm-tools` family of crates. Additionally Wasmtime is updated with the
same upstream versions of the `wasm-tools` crates. Put together this
updates everything to the most recent version of the component model
with optional URLs encoded and an updated version of Wasmtime.
Part of this transition is the removal of
`wit_component::ComponentInterfaces` which is plumbed through as part of
this commit.
* Update to wasmtime main
* Fix demo build
* Update wasm-tools binary
* Fix cli build
* Install wasm-tools from crates.io, not git
This commit removes the host Python generation from this project. The
support here has all moved upstream to
https://github.com/bytecodealliance/wasmtime-py and is now supported
through:
python -m wasmtime.bindgen my-component.wasm --out-dir ./my-component
Some documentation is in the upstream `README.md` about how to use.
Necessary tests have been mirrored to the upstream repository to the
best of my ability, and this is intended to represent an inflection
point where the growth of the Python generator is no longer tied up with
all the other languages in this repository. This is a step further along
the path of #395.
* add guest binding generator for TeaVM/Java
This generator is for [teavm-wasi](https://github.com/fermyon/teavm-wasi),
a friendly fork of TeaVM with support for WASI and the component model
canonical ABI.
All the basic features are supported except for async functions, resources,
and multi-value return. The unsupported features shouldn't be hard to add
in follow-up commits.
[Maven](https://maven.apache.org/) is required to run the tests.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* add `actions/setup-java` to `test` CI job for `gen-guest-teavm-java`
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* add Java runtime test driver; add Java to demo
So far the only runtime test exercised is the smoke one. I'll follow this up
with other test implementations.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* use cmd to run mvn on Windows
We need to do this since Maven is installed as `mvn.cmd` on Windows.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
This commit removes `wasmlink`, `wasmlink-cli`, and `test-modules` from the
`wit-bindgen` repo.
`wasmlink` is based on an outdated module linking proposal and predates any of
the work on the component model. As such, it never really graduated beyond a
useful demo of how multiple core WebAssembly modules might exchange interface
types.
Now that the work on the component model is progressing, the underlying
wasm-tools no longer support the module linking proposal; therefore, `wasmlink`
is becoming a maintenance burden.
As the author of `wasmlink`, I think the time has come to remove it from the
tree. Any parties that need a copy of `wasmlink` for whatever reason may fork
it prior to this commit being merged.
Currently `wasmlink::Module` only supports reading an interface from a
custom section (unused) or from a path to a witx file.
In preparation for some upcoming work, this commit implements creating a
`wasmlink::Module` that exports more than one `witx2::Interface`.
`Module::new` now checks that given interfaces are not in conflict (e.g.
have a function/resource of the same name) and also verify that the
module actually does export those interfaces.
Also fixed some clippy lints in wasmlink.
* Initial commit of a wasmtime-py bindings generator
Currently supports everything necessary to run `host.witx`, but there
are known missing holes:
* Endianness isn't handled
* Probably not as efficient as it could be
* Legacy witx features aren't supported
This generator is intended for host embeddings of Python using the
`wasmtime` package on PyPI. This will generate Python with annotated
types for usage with `mypy` and other type checkers as well (and `mypy`
is a bit part of the tests for all of this). The idioms generated here
are a sort of blend of JS idioms where ownership doesn't matter and
Wasmtime idioms where types like the `Store` must be passed around. This
is intended to feel like idiomatic Python, but being someone who hasn't
written a lot of Python I'm sure there's room for improvement here and
there, so this is just a first pass.
* Python syntax highlighting in demo
* Document markdown in readme
* Install Python on GHA
* js: Use objects/methods for resources with methods
This commit generates more idiomatic JS when resources have methods and
static functions associated with them. Resources exported to JS now have
an object-like interface to remove the need to deal with the original
wasm object. Resources imported from JS have Typescript interfaces
generated which allows using method-call syntax to invoke methods. Note
that static methods on resources imported from JS are still freestanding
methods on the import object to prevent the generated glue code from
having to import a named type.
cc #38
* rust-wasm: Use objects/methods for resources with methods
This is the same as the previous commit, except for the rust-wasm
generator. This enables imports and exports to use associated functions
and methods where appropriate to be more idiomatic within Rust. Note
that this only applies to resources with functions associated with them.
This change restructures how exports of a rust-wasm module are modeled.
Previously a trait was generated and a function was called to return a
singleton instance of the trait, but now the trait's methods no longer
take `&self` and it's expected that a zero-sized type is used to
implement the trait. Separate traits are generated for each resource
with methods as well.
Resource traits (for exports) take `&self` for methods, and otherwise
work with `Handle<T>` types for all other handle types (e.g. results or
other arguments).
cc #38
* Fix & move demo
We've got two demos now, so make it clear that `crates/demo` is the
witx-bindgen demo. Additionally move the build script into that
directory.
* Fix wasmlink demo
* wip
* Copy witx files into `test-modules` directory
For now the intention is that the `tests` directory is primarily tests
for code generators, where adding more witx files there is intended to
exercise more cases in code generators, not necessarily having backing
code somewhere implementing it.
I copied the existing `*.witx` files into the `test-modules` directories
since they all assume that the `*.witx` has a particular shape. This
is a bit of duplication but allows the two to go out of sync so new
codegen tests don't have to also update the tests for linking.
* flags & C
* Pass flags by value in C
* Basic list support for C
* Get things compiling again
* More C bits working
* Some C renamings
* Test all imports for C
* Test all the C exports
* Flesh out the C generator for exports
Everything should now be wired up and the same Rust/JS host code can
consume, transparently, a C source and a Rust source module.
* Fill out C imports test
* Add tests for C buffers
* Fix rust import test
* Install wasi sdk on CI
* Remove unnecessary comments
* Remove warnings from C generator
* Uncomment some TODO
* Show stderr of failed wasmlink tests
* Adjust wasmlink for new realloc parameter order
* Different tarball per-platform
* Add C to README
* Fix test expectation
Also fix a number of issues while I'm at it:
* Rust macros use `include_str!` to depend on the input `*.witx` files
* Fix variables of the name `async` in Rust
* Fix variables of the name `import` in JS
* Move the demo to TypeScript to be more resilient to witx changes
* Spice up the default demo slightlyj
* Beginnings of a JS generator
* JS fixes and move wasmtime test around
* Fix warnings
* Add a shim of a demo
* Fix CI
* Fix CI again
* Disable tests on demo
* More support for JS exports
* Add a small demo
* tabs
* add build script
* fix ci
* Tweak JS slightly
* Fix handling zero-length strings
* Fix encoding multibyte char strings in JS
* Generate more reasonably formatted Rust code by default
If we can't run rustfmt on the web let's just do a poor-man's rustfmt
* More newlines
* only deploy on main
* Add JS to readme
