`(task:)Cancel` is now `<suspends>`. If the target task, or one of its descendents, is on the native stack (e.g. because it is, started, or resumed the current task, or because it was resumed from an `Await` or `Cancel` alongside such a task), the canceling task yields.
When an individual canceled task has no more children, its call stack is unwound. Rather than change the calling convention for `<suspends>` functions, unwinding control flow edges are encoded as a table in `VProcedure`. Unwinding finds the innermost `defer` block or task boundary by walking the stack, and `defer` blocks entered this way end with a new `ResumeUnwind` instruction.
Because cancellation interacts with the task parent/child relation, this change includes `branch` as a simple way to write tests. The remaining structured concurrency constructs will come in a future change.
Native `<suspends>` functions now return a new `FVerseResult` object rather than a bare `ECoroutineResult`. Today, this simplifies the mechanism for transmitting the return value. It was also originally intended as a way for native functions to initiate cancellation; this turned out to be unnecessary in the current implementation, but enables a future `CancelIfRequested` API.
#rb saam.barati, Tim.Smith
#okforversepublic
[CL 33307997 by russell johnston in ue5-main branch]
#rb Russell.Johnston Tim.Smith
This patch implements deep mutability in the new VM. To do this, I implemented the following:
- Renamed IndexSet to CallSet for name parity with call.
- Made it so we can freeze and melt structs. This wasn't implemented before. To melt a struct, we create a new emergent type where all fields live in the object itself. The melted struct will use this emergent type. We also cache this resulting emergent type on the source emergent type so that freezing all structs with emergent type E1 always produce and object with emergent type E2.
- Emit freeze/melt inside the bytecode generator at the right times. This patch emits freeze/melt unoptimally. We'll need to do FoX in the future to get better perf.
- We pattern match against the AST to match the section of AST that corresponds to a successive list of accesses that are all part of the same deep mutation. We use this to drive codegen for deep mutability.
- Refactor our intrinsics so the code for `set ... = ` can be shared with things like `set ... +=`.
[CL 32869234 by saam barati in ue5-main branch]
- Removes VUTF8String and replaces its usage with VArray's of Char8's.
- Reworks VUniqueString to derive from VArray
- Re-enables majority of string related verse-tests for VM
Minor:
- Updated some ToStringImpl for VFalse and VFunction
#okforversepublic
#rb saam.barati, Tim.Smith
#rb tim.smith
[CL 32803034 by jared cotton in ue5-main branch]
Defer blocks run for one of several reasons: normal block exit, exit via break or return, or (not yet implemented) task cancellation. To accommodate this, defer bytecode accepts a jump target index and switches on it after running. This index is stored in a register allocated for the defer block, which is sufficient because defers are not re-entrant. The register and switch are elided when the block runs only on the normal exit path.
The switch instruction has a variadic label operand. To simplify label fixup, all variadic operands are moved from `TArray` fields in the instructions to a shared trailing array in `VProcedure`. (Variadic operands to suspension captures remain `TArray`s.)
Early exit may cross several defer blocks on the way to its eventual jump target. Each defer block wraps these jump targets in a thunk, which passes the index for the wrapped target to the defer block. Exit control flow thus executes a chain of these thunks, ending with the actual exit jump when it reaches the appropriate scope.
#rb Markus.Breyer, Tim.Smith
[CL 32670628 by russell johnston in ue5-main branch]
Bind the `task` class to `VTask` with a native `Await` method. For tasks created before this binding, `VTask` uses a trivial emergent type and shape, so code that runs in this context (i.e. module top-levels) does not support structured concurrency.
Each `sync` arm spawns a new task, and the overall expression `Await`s each of them in turn to collect their results. These tasks may need to remain active when the outer `sync` expression suspends, which conflicts with the way lenient suspension forcibly suspends child tasks - to resolve this, suspenion no longer affects child tasks (in fact child tasks are no longer tracked at all) and instead `EndTask` (really any time the current task changes) must check for an already-suspended `YieldTask`.
Also adjust the bytecode of `for` loops so that the body runs outside of any failure context, rather than forwarding failures. This enables `<suspends>` calls in `for` loop bodies.
#rb Markus.Breyer
[CL 32277546 by russell johnston in ue5-main branch]
Replace `TVerseFunction` and its `FProperty` with a `VRestValue`.
Unify `TVerseFunction` and `FInteropUtils::TryInvoke...` for calling methods vs module/interface functions. (This includes fixing named non-defaulted parameters in the `TVerseFunction` path, apparently previous unused.)
Wrap the `verse::task*` result of `<native_callable>` APIs in `FVerseTask`; remove `Cancel` from `TVerseCall` (see SOL-6056).
Switch to decorated but unmangled names in VNI-generated binding glue.
Switch to decorated method names in VerseVM to match the binding glue.
Run `UVerseVMClass` CDO initialization after binding glue computes class layout.
Always create a `UObject` for `<native>` classes, now that their methods are bound.
#rb markus.breyer, tim.smith
#okforversepublic
[CL 32004566 by russell johnston in ue5-main branch]
#rb russell.johnston
#okforversepublic
Top-level code is allowed to have mostly arbitrary expressions in it. For example, you could have `X:int = 1+2` at the top level. Prior to this patch, we had a janky AST interpreter that pattern matched code at the top level. It only worked for things that were constant to the degree it understood. However, it's cleaner to just generate bytecode for top-level code that uses the normal interperter and binds evaluation results to variables.
[CL 31833870 by saam barati in ue5-main branch]
#okforversepublic
VerseVM:
- named & default params
- variadic immediate support in VVM code generator
[CL 31679794 by shawn mcgrath in ue5-main branch]
Added support for Visit template specialization. This allows for C++ structures to have visitors defined without having to modify the visitor objects. Work in progress.
#rnx
#rb yiliang.siew
[CL 31288348 by tim smith in ue5-main branch]
A new VTask object tracks a chain of the VFrames, from the point where the task was spawned to the point where it suspended. This task boundary is recorded by a pair of BeginTask/EndTask instructions. Individual <suspends> function calls in the new VM do not have distinct tasks, and <suspends> does not change how bytecode is generated.
Native functions may suspend the current task by returning a new FOpResult of kind Yield, at which point the current PC is saved to the VTask, its VFrames are detached from the stack, and execution continues in the parent task. Native code may resume a task with a new VTask::ResumeInTransaction API, which re-attaches these VFrames to the top of the stack and continues execution from the saved PC.
VNI functions continue to use the verse::continuation API, which now wraps a VTask pointer and thus must be accounted for during GC tracing.
#rb Markus.Breyer, saam.barati
#okforversepublic
[CL 30978163 by russell johnston in ue5-main branch]
- Added OpCodes melt/freeze which are how deep-copying of vars will be done. ie `VarSet(Melt(Value))` and `Freeze(VarGet(Var))`
- melt suspends upon encountering a placeholder
- freeze errors on encountering a placeholder. This may change depending on whether the Verse-Verifier decides to allow this or not.
- note: We use a VCell header bit to identify types which are deeply mutable and thus require melt/freeze operations (this may be moved into these types emergent types)
- Re-worked VMap classes so now majority of code lives inside VMapBase via templating
#rb saam.barati
[CL 30688090 by jared cotton in ue5-main branch]
#rb yiliang.siew
This patch:
- Makes map construction pick the right values in the right order when constructing a map with duplicate keys
- Makes map comparison ordered
- Implements ConcatenateMaps as an intrinsic function
- Makes enum hashable
[CL 30365816 by saam barati in ue5-main branch]
Split VMap into mutable and immutable versions
(preflight failure looked spurious... ie timeout on cook worker. Ready to backout if this is a real issue somehow caused by my CL but thought this unlikely)
[CL 29994701 by jared cotton in ue5-main branch]
SOL-4828 - "Split container types between being mutable and immutable"
- Changed VArray to be an immutable array and created VMutableArray
- Removed VTuple and replaced its usages with VArray. VArray will be updated to take an emergent type in its construction later to allow for defining whether it is an array or a tuple for casting purposes.
- Added `Aux Allocation` logic to GC/Barriers which marks but does not add to the MarkStack. This is used for allocating the VMutableArray's buffer of TWriteBarrier<VValue>'s.
#rb Kirill.Zorin
#rb Saam.Barati
[CL 29866547 by jared cotton in ue5-main branch]
Represent class initialization logic with a new `VConstructor` type, holding a sequence of interleaved fields and blocks. At class construction time, flatten these sequences such that class bodies run in order, and subclasses run before superclasses. At object construction time, extract a sequence of `VProcedure`s representing the non-overridden field initializers and blocks to invoke.
Give these initializers and blocks (as well as member functions) access to their `Self` object via a `ParentScope` capture in `VFunction`, copied to register 0 at call time. Rather than close over `Self` at object construction time, do it at field load time, so these members can be shared between all instances of a class as `VProcedures`, either in the `VConstructor` or `VShape`. In the future, the `ParentScope` field for non-member functions may instead point to an environment.
#rb saam.barati, yiliang.siew
[CL 29530839 by russell johnston in ue5-main branch]
#rb yiliang.siew
We're soon gonna be storing to VValue/pointer fields in the bytecode stream that the GC will scan and the mutator will write to. Let's 8-byte align so we don't see tearing.
[CL 28779525 by saam barati in ue5-main branch]
- Opcodes now support having immediate operands.
- Fixes the indices not being generated correctly for object shapes.
- New opcodes `NewClass`, `NewObject`, `LoadField` and `UnifyField` for working with classes and objects.
- Some `const` cleanup of arguments.
- Also update UBT to disable the "first include must match in terms of filename" check on `VerseVM`, since it brings us no benefits and actively interferes with the way we do inline versions of our headers.
#rnx
#rb Saam.Barati
#rb Phil.Pizlo
[CL 28541942 by yiliang siew in ue5-main branch]
VerseCLR can now compile verse code with maps, and iterations over maps, to the new VM.
Fix unrelated bug triggered by nested for-iterations over arrays, returning an array of arrays.
#rb Saam.Barati
#rnx
[CL 27582449 by niklas rojemo in ue5-main branch]
#rb markus.breyer
This patch makes it so calls with multiple arguments don't require a tuple allocation. We just allow functions to have multiple arguments and be called normally. However, when such functions are invoked with a tuple, we'll unbox the tuple into the argument slots.
We also allow functions with single-param-as-tuple to be invoked with multiple arguments. And such functions will box the arguments into a tuple when being called.
A function with N arguments now take up the first N register slots in the frame.
This patch also fills out some more of the bytecode generator to make some tests I wrote work.
[CL 27510491 by saam barati in ue5-main branch]
