Block patches:
- drop block/io write notifiers
- qemu-iotests enhancements to make debugging easier
- rbd parsing fix
- HMP qemu-io fix (for iothreads)
- mirror job cancel relaxation (do not cancel in-flight requests when a
READY mirror job is canceled with force=false)
- document qcow2's data_file and data_file_raw features
- fix iotest 297 for pylint 2.8
- block/copy-on-read refactoring
# gpg: Signature made Fri 14 May 2021 17:43:40 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2021-05-14:
write-threshold: deal with includes
test-write-threshold: drop extra TestStruct structure
test-write-threshold: drop extra tests
block/write-threshold: drop extra APIs
test-write-threshold: rewrite test_threshold_(not_)trigger tests
block: drop write notifiers
block/write-threshold: don't use write notifiers
qemu-iotests: fix pylint 2.8 consider-using-with error
block/copy-on-read: use bdrv_drop_filter() and drop s->active
Document qemu-img options data_file and data_file_raw
qemu-iotests: fix case of SOCK_DIR already in the environment
qemu-iotests: let "check" spawn an arbitrary test command
qemu-iotests: move command line and environment handling from TestRunner to TestEnv
qemu-iotests: allow passing unittest.main arguments to the test scripts
qemu-iotests: do not buffer the test output
mirror: stop cancelling in-flight requests on non-force cancel in READY
monitor: hmp_qemu_io: acquire aio contex, fix crash
block/rbd: Add an escape-aware strchr helper
iotests/231: Update expected deprecation message
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The default "alabaster" sphinx theme has a couple shortcomings:
- the navbar moves along the page
- the search bar is not always at the same place
- it lacks some contrast and colours
The "rtd" theme from readthedocs.org is a popular third party theme used
notably by the kernel, with a custom style sheet. I like it better,
perhaps others do too. It also simplifies the "Edit on Gitlab" links.
Tweak a bit the custom theme to match qemu.org style, use the
QEMU logo, and favicon etc.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Message-Id: <20210323115328.4146052-1-marcandre.lureau@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Implementing FUSE exports required no changes to the storage daemon, so
we forgot to document them there. Considering that both NBD and
vhost-user-blk exports are documented in its man page (and NBD exports
in its --help text), we should probably do the same for FUSE.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210217115844.62661-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
This switches qemu-img from a QemuOpts-based parser for --object to
user_creatable_process_cmdline() which uses a keyval parser and enforces
the QAPI schema.
Apart from being a cleanup, this makes non-scalar properties accessible.
As a side effect, fix wrong exit codes in the object parsing error path
of 'qemu-img compare'. This was broken in commit 334c43e2c3 because
&error_fatal exits with an exit code of 1, while it should have been 2.
Document that exit code 0 is also returned when just requested help was
printed instead of comparing images. This is preexisting behaviour that
isn't changed by this patch, though another instance of it is added with
'--object help'.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
World-writeable directories have security issues. Avoid showing them in
the documentation since someone might accidentally use them in
situations where they are insecure.
There tend to be 3 security problems:
1. Denial of service. An adversary may be able to create the file
beforehand, consume all space/inodes, etc to sabotage us.
2. Impersonation. An adversary may be able to create a listen socket and
accept incoming connections that were meant for us.
3. Unauthenticated client access. An adversary may be able to connect to
us if we did not set the uid/gid and permissions correctly.
These can be prevented or mitigated with private /tmp, carefully setting
the umask, etc but that requires special action and does not apply to
all situations. Just avoid using /tmp in examples.
Reported-by: Richard W.M. Jones <rjones@redhat.com>
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20210301172728.135331-3-stefanha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Daemons often have a --pidfile option where the pid is written to a file
so that scripts can stop the daemon by sending a signal.
The pid file also acts as a lock to prevent multiple instances of the
daemon from launching for a given pid file.
QEMU, qemu-nbd, qemu-ga, virtiofsd, and qemu-pr-helper all support the
--pidfile option. Add it to qemu-storage-daemon too.
Reported-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20210302142746.170535-1-stefanha@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
On Linux, the 'security.capability' xattr holds a set of
capabilities that can change when an executable is run, giving
a limited form of privilege escalation to those programs that
the writer of the file deemed worthy.
Any write causes the 'security.capability' xattr to be dropped,
stopping anyone from gaining privilege by modifying a blessed
file.
Fuse relies on the daemon to do this dropping, and in turn the
daemon relies on the host kernel to drop the xattr for it. However,
with the addition of -o xattrmap, the xattr that the guest
stores its capabilities in is now not the same as the one that
the host kernel automatically clears.
Where the mapping changes 'security.capability', explicitly clear
the remapped name to preserve the same behaviour.
This bug is assigned CVE-2021-20263.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Vivek Goyal <vgoyal@redhat.com>
When we first converted our documentation to Sphinx, we split it into
multiple manuals (system, interop, tools, etc), which are all built
separately. The primary driver for this was wanting to be able to
avoid shipping the 'devel' manual to end-users. However, this is
working against the grain of the way Sphinx wants to be used and
causes some annoyances:
* Cross-references between documents become much harder or
possibly impossible
* There is no single index to the whole documentation
* Within one manual there's no links or table-of-contents info
that lets you easily navigate to the others
* The devel manual doesn't get published on the QEMU website
(it would be nice to able to refer to it there)
Merely hiding our developer documentation from end users seems like
it's not enough benefit for these costs. Combine all the
documentation into a single manual (the same way that the readthedocs
site builds it) and install the whole thing. The previous manual
divisions remain as the new top level sections in the manual.
* The per-manual conf.py files are no longer needed
* The man_pages[] specifications previously in each per-manual
conf.py move to the top level conf.py
* docs/meson.build logic is simplified as we now only need to run
Sphinx once for the HTML and then once for the manpages5B
* The old index.html.in that produced the top-level page with
links to each manual is no longer needed
Unfortunately this means that we now have to build the HTML
documentation into docs/manual in the build tree rather than directly
into docs/; otherwise it is too awkward to ensure we install only the
built manual and not also the dependency info, stamp file, etc. The
manual still ends up in the same place in the final installed
directory, but anybody who was consulting documentation from within
the build tree will have to adjust where they're looking.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210115154449.4801-1-peter.maydell@linaro.org
Document the qemu-storage-daemon tool. Most of the command-line options
are identical to their QEMU counterparts. Perhaps Sphinx hxtool
integration could be extended to extract documentation for individual
command-line options so they can be shared. For now the
qemu-storage-daemon simply refers to the qemu(1) man page where the
command-line options are identical.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201209103802.350848-3-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Split the documentation of the qemu-pr-helper binary into the tools
manual, and give it a manpage like our other standalone executables.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Allow the server to expose an additional metacontext to be requested
by savvy clients. qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using block-export-add.
qemu as client is hacked into viewing the key aspects of this new
context by abusing the already-experimental x-dirty-bitmap option to
collapse all depths greater than 2, which results in a tri-state value
visible in the output of 'qemu-img map --output=json' (yes, that means
x-dirty-bitmap is now a bit of a misnomer, but I didn't feel like
renaming it as it would introduce a needless break of back-compat,
even though we make no compat guarantees with x- members):
unallocated (depth 0) => "zero":false, "data":true
local (depth 1) => "zero":false, "data":false
backing (depth 2+) => "zero":true, "data":true
libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-11-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Block layer patches:
- qcow2: Skip copy-on-write when allocating a zero cluster
- qemu-img: add support for rate limit in qemu-img convert/commit
- Fix deadlock when deleting a block node during drain_all
# gpg: Signature made Tue 27 Oct 2020 15:14:07 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
block: End quiescent sections when a BDS is deleted
qcow2: Skip copy-on-write when allocating a zero cluster
qcow2: Report BDRV_BLOCK_ZERO more accurately in bdrv_co_block_status()
qemu-img: add support for rate limit in qemu-img convert
qemu-img: add support for rate limit in qemu-img commit
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The mapping rule system implemented in the last few patches is
extremely flexible, but not easy to use. Add a simple
'map' type as a sprinkling of sugar to make it easy.
e.g.
-o xattrmap=":map::user.virtiofs.:"
would be sufficient to prefix all xattr's
or
-o xattrmap=":map:trusted.:user.virtiofs.:"
would just prefix 'trusted.' xattr's and leave
everything else alone.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-6-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Add an option to define mappings of xattr names so that
the client and server filesystems see different views.
This can be used to have different SELinux mappings as
seen by the guest, to run the virtiofsd with less privileges
(e.g. in a case where it can't set trusted/system/security
xattrs but you want the guest to be able to), or to isolate
multiple users of the same name; e.g. trusted attributes
used by stacking overlayfs.
A mapping engine is used with 3 simple rules; the rules can
be combined to allow most useful mapping scenarios.
The ruleset is defined by -o xattrmap='rules...'.
This patch doesn't use the rule maps yet.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-2-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Peter Maydell
Connor KuehlandMax Reitz
Marc-André Lureau
Kevin Wolf
Stefan HajnocziandKevin Wolf
Dr. David Alan Gilbert
Daniel P. BerrangéandPaolo Bonzini
Eric Blake
Stefan WeilandPaolo Bonzini
ZhenguiandKevin Wolf