All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.
Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides. QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-14-armbru@redhat.com>
When a command's 'data' is an object, its doc comment describes the
arguments defined there. When 'data' names a type, the doc comment
does not describe arguments. Instead, the doc generator inserts a
pointer to the named type.
An event's doc comment works the same.
We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked. Works because the
check only cares for the implicit argument type's members.
Not only is this needlessly hard to understand, it actually falls
apart in two cases:
* When 'data' is empty, there is nothing to forward to, and the doc
comment remains unchecked. Demonstrated by test doc-bad-event-arg.
* When 'data' names a type, we can't forward, as the type has its own
doc comment. The command or event's doc comment remains unchecked.
Demonstrated by test doc-bad-boxed-command-arg.
The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation". That
fix was incomplete.
To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
An object type's doc comment describes the type's members, less the
ones defined in a named base type. Cases:
* Struct: the members are defined in 'data' and inherited from 'base'.
Since the base type cannot be implicit, the doc comment describes
just 'data'.
* Simple union: the only member is the implicit tag member @type, and
the doc comment describes it.
* Flat union with implicit base type: the members are defined in
'base', and the doc comment describes it.
* Flat union with named base type: the members are inherited from
'base'. The doc comment describes no members.
Before we can check a doc comment with .check_doc(), we need
.connect_doc() connect each of its "argument sections" to the member
it documents.
For structs and simple unions, this is straightforward: the members in
question are in .local_members, and .connect_doc() connects them.
For flat unions with a named base type, it's trivial: .local_members
is empty, and .connect_doc() does nothing.
For flat unions with an implicit base type, it's tricky. We have
QAPISchema._make_implicit_object_type() forward the union's doc
comment to the implicit base type, so that the base type's
.connect_doc() connects the members. The union's .connect_doc() does
nothing, as .local_members is empty.
Dirt effect: we check the doc comment twice, once for the union type,
and once for the implicit base type.
This is needlessly brittle and hard to understand. Clean up as
follows. Make the union's .connect_doc() connect an implicit base's
members itself. Do not forward the union's doc comment to its
implicit base type.
Requires extending .connect_doc() so it can work with a doc comment
other than self.doc. Add an optional argument for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-11-armbru@redhat.com>
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member. This is because we neglect
to call self.doc.check() for enumeration types. Messed up in
816a57cd6e "qapi: Fix detection of bogus member documentation". Fix
it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-10-armbru@redhat.com>
Splitting documentation checking off the .check() methods makes them a
bit more focused, which is welcome, as some of them are pretty big.
It also prepares the ground for the following commits.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-9-armbru@redhat.com>
QAPISchemaGenDocVisitor.visit_command() duplicates texi_entity() for
its boxed arguments case. The previous commit added another copy in
.visit_event().
Replace texi_entity() by texi_type() and texi_msg(). Use texi_msg()
for the boxed arguments case as well.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-8-armbru@redhat.com>
Generate a reference "Arguments: the members of ...", just like we do
for commands since commit c2dd311cb7 "qapi2texi: Implement boxed
argument documentation".
No change to generated QMP documentation; we don't yet use boxed
events outside tests/.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-7-armbru@redhat.com>
Similarly to features for struct types introduce the feature flags also
for commands. This will allow notifying management layers of fixes and
compatible changes in the behaviour of a command which may not be
detectable any other way.
The changes were heavily inspired by commit 6a8c0b5102.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-3-armbru@redhat.com>
The QAPI code generator clocks in at some 3100 SLOC in 8 source files.
Almost 60% of the code is in qapi/common.py. Split it into more
focused modules:
* Move QAPISchemaPragma and QAPISourceInfo to qapi/source.py.
* Move QAPIError and its sub-classes to qapi/error.py.
* Move QAPISchemaParser and QAPIDoc to parser.py. Use the opportunity
to put QAPISchemaParser first.
* Move check_expr() & friends to qapi/expr.py. Use the opportunity to
put the code into a more sensible order.
* Move QAPISchema & friends to qapi/schema.py
* Move QAPIGen and its sub-classes, ifcontext,
QAPISchemaModularCVisitor, and QAPISchemaModularCVisitor to qapi/gen.py
* Delete camel_case(), it's unused since commit e98859a9b9 "qapi:
Clean up after recent conversions to QAPISchemaVisitor"
A number of helper functions remain in qapi/common.py. I considered
moving the code generator helpers to qapi/gen.py, but decided not to.
Perhaps we should rewrite them as methods of QAPIGen some day.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-7-armbru@redhat.com>
[Add "# -*- coding: utf-8 -*-" lines]
The next commit will split up qapi/common.py. gen_enum() needs
QAPISchemaEnumMember, and that's in the way. Move it to qapi/types.py
along with its buddy gen_enum_lookup().
Permit me a short a digression on history: how did gen_enum() end up
in qapi/common.py? Commit 21cd70dfc1 "qapi script: add event support"
duplicated qapi-types.py's gen_enum() and gen_enum_lookup() in
qapi-event.py. Simply importing them would have been cleaner, but
wasn't possible as qapi-types.py was a program, not a module. Commit
efd2eaa6c2 "qapi: De-duplicate enum code generation" de-duplicated by
moving them to qapi.py, which was a module.
Since then, program qapi-types.py has morphed into module types.py.
It's where gen_enum() and gen_enum_lookup() started, and where they
belong.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-6-armbru@redhat.com>
The frontend can't be run more than once due to its global state.
A future commit will want to do that.
The only global frontend state remaining is accidental:
QAPISchemaParser.__init__()'s parameter previously_included=[].
Python evaluates the default once, at definition time. Any
modifications to it are visible in subsequent calls. Well-known
Python trap. Change the default to None and replace it by the real
default in the function body. Use the opportunity to convert
previously_included to a set.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-4-armbru@redhat.com>
The frontend can't be run more than once due to its global state.
A future commit will want to do that.
Recent commit "qapi: Move context-sensitive checking to the proper
place" got rid of many global variables already, but pragma state is
still stored in global variables (that's why a pragma directive's
scope is the complete schema).
Move the pragma state to QAPISourceInfo.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-3-armbru@redhat.com>
Commit bc52d03ff5 "qapi: Make doc comments optional where we don't
need them" made scripts/qapi2texi.py fail[*] unless the schema had
pragma 'doc-required': true. The stated reason was inability to cope
with incomplete documentation.
When commit fb0bc835e5 "qapi-gen: New common driver for code and doc
generators" folded scripts/qapi2texi.py into scripts/qapi-gen.py, it
turned the failure into silent suppression.
The doc generator can cope with incomplete documentation now. I don't
know since when, or what the problem was, or even whether it ever
existed.
Drop the silent suppression.
[*] The fail part was broken, fixed in commit e8ba07ea9a.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-2-armbru@redhat.com>
The following statement produces a SyntaxWarning with Python 3.8:
if len(format) is 0:
scripts/tracetool/__init__.py:459: SyntaxWarning: "is" with a literal. Did you mean "=="?
Use the conventional len(x) == 0 syntax instead.
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191010122154.10553-1-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
The `make efi` target added by 536d2173 is built from the roms/edk2
submodule, which in turn relies on additional submodules nested under
roms/edk2.
The make-release script currently only pulls in top-level submodules,
so these nested submodules are missing in the resulting tarball.
We could try to address this situation more generally by recursively
pulling in all submodules, but this doesn't necessarily ensure the
end-result will build properly (this case also required other changes).
Additionally, due to the nature of submodules, we may not always have
control over how these sorts of things are dealt with, so for now we
continue to handle it on a case-by-case in the make-release script.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Bruce Rogers <brogers@suse.com>
Cc: qemu-stable@nongnu.org # v4.1.0
Reported-by: Bruce Rogers <brogers@suse.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Message-Id: <20190912231202.12327-2-mdroth@linux.vnet.ibm.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The low bits are 1 if the control must be one, the high bits
are 1 if the control can be one. Correct the variable names
as they are very confusing.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file. Lazy.
Change QAPISchema.__init__() to take a file name instead of a file
object. Move the open code from _include() to __init__(), so it's
used for the main schema file, too.
Move the read into the try for good measure, and rephrase the error
message.
Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source". Make QAPISourceInfo cope
with fname=None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-27-armbru@redhat.com>
Markus Armbruster
Paolo Bonzini
Wei YangandPaolo Bonzini
Peter KrempaandMarkus Armbruster
Stefan Hajnoczi
Michael RothandPhilippe Mathieu-Daudé