Symbolization is a costly operation: do it only when requested. Rework
the C API to make this possible. For now, do it automatically in the
Java API to avoid breaking existing Java programs: this will fix the
performance situation in the other C-based APIs.
Now that the Python DSL is gone, must line number hiding is
counter-productive: Python line numbers are not expected to show up in
test outputs in the general case, and hiding them hindes debugging when
there is a Python crash in a testcase. Rework the corresponding output
refiners so that this is done only when actually needed.
Also rework lkt_compile and lkt_build_and_run test drivers so that the
language spec configuration bits are stored in the testcase's test.yaml
file directly: no need to add glue code in various places (test driver,
lkt_compile.py/lkt_build_and_run.py scripts) to support a new kind of
language spec configuration.
Whether unparsers are generated or not has implications on legality
checks for the grammar, so it makes sense for this to be a property of
the grammar itself: introduce an Lkt annotation as well as a
langkit.parsers.Grammar argument for it, and remove the corresponding
CompileCtx argument.
Also rename "generate_unparser" to "generate_unparsers" for consistency
across the codebase.
Transition all relevant testcases to it. In the process, port
docstrings/comments from test.py files to expected_concrete_syntax.lkt
(now test.lkt) sources and fix stylechecks issues there.
This makes the convention consistent with the DSL, and avoids
workarounds for conflicts with Lkt keywords: in Libadalang, the Null
token can stay Null, instead of null_tok (no API breakage needed).
That'll allow us to go from (thin_symbol, sym_table) to symbol, and vice
versa, where with the old representation there was no other way to go
from symbol to thin than redoing a full-lookup.
That'll help us transition to thin-symbol storage everywhere.
Future work on Lkt will use this compiled type so that properties can
return precisely located errors on string literals.
Note that the exposition of this new compiled type to the OCaml and Java
APIs is incomplete, and is deferred to future developments.
