Our documentation-related tools are spread out over various directories;
several are buried in the scripts/ dumping ground. That makes them harder
to discover and harder to maintain.
Recent work has started accumulating our documentation-related tools in
/tools/docs. This series nearly completes that task, moving most of the
rest of our various utilities there, hopefully fixing up all of the
relevant references in the process.
The one exception is scripts/kernel-doc; that move turned up some other
problems, so I have dropped it until those are ironed out.
At the end, rather than move the old, Perl kernel-doc, I simply removed it.
Move this tool out of scripts/ to join the other documentation tools; fix
up a couple of erroneous references in the process.
It's worth noting that this script will fail badly unless one has a
PYTHONPATH referencing scripts/lib/abi.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The scripts for managing the features docs are found in three different
directories; unite them all under tools/docs and update references as
needed.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Quoth Mauro:
This series should probably be called:
"Move the trick-or-treat build hacks accumulated over time
into a single place and document them."
as this reflects its main goal. As such:
- it places the jobserver logic on a library;
- it removes sphinx/parallel-wrapper.sh;
- the code now properly implements a jobserver-aware logic
to do the parallelism when called via GNU make, failing back to
"-j" when there's no jobserver;
- converts check-variable-fonts.sh to Python and uses it via
function call;
- drops an extra script to generate man pages, adding a makefile
target for it;
- ensures that return code is 0 when PDF successfully builds;
- about half of the script is comments and documentation.
I tried to do my best to document all tricks that are inside the
script. This way, the docs build steps is now documented.
It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.
Such script was written in a way that it can be called either
directly or via a Makefile. Running outside Makefile is
interesting specially when debug is needed. The command line
interface replaces the need of having lots of env vars before
calling sphinx-build:
$ ./tools/docs/sphinx-build-wrapper --help
usage: sphinx-build-wrapper [-h]
[--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
[--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
[-j JOBS] [-i] [-V [VENV]]
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Kernel documentation builder
positional arguments:
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Documentation target to build
options:
-h, --help show this help message and exit
--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
Specific directories to build
--conf CONF Sphinx configuration file
--builddir BUILDDIR Sphinx configuration file
--theme THEME Sphinx theme to use
--css CSS Custom CSS file for HTML/EPUB
--paper {,a4,letter} Paper size for LaTeX/PDF output
-v, --verbose place build in verbose mode
-j, --jobs JOBS Sets number of jobs to use with sphinx-build
-i, --interactive Change latex default to run in interactive mode
-V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)
the only mandatory argument is the target, which is identical with
"make" targets.
The call inside Makefile doesn't use the last four arguments. They're
there to help identifying problems at the build:
-v makes the output verbose;
-j helps to test parallelism;
-i runs latexmk in interactive mode, allowing to debug PDF
build issues;
-V is useful when testing it with different venvs.
When used with GNU make (or some other make which implements jobserver),
a call like:
make -j <targets> htmldocs
will make the wrapper to automatically use POSIX jobserver to claim
the number of available job slots, calling sphinx-build with a
"-j" parameter reflecting it. ON such case, the default can be
overriden via SPHINXDIRS argument.
Visiable changes when compared with the old behavior:
When V=0, the only visible difference is that:
- pdfdocs target now returns 0 on success, 1 on failures.
This addresses an issue over the current process where we
it always return success even on failures;
- it will now print the name of PDF files that failed to build,
if any.
In verbose mode, sphinx-build-wrapper and sphinx-build command lines
are now displayed.
C domain supports a ".. c:namespace::" tag that allows setting a
symbol namespace. This is used within the kernel to avoid warnings
about duplicated symbols. This is specially important for syscalls,
as each subsystem may have their own documentation for them.
This is specially true for ioctl.
When such tag is used, all C domain symbols have c++ style,
e.g. they'll become "{namespace}.{reference}".
Allow specifying C namespace at the exception files, avoiding
the need of override rules for every symbol.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <cc27ec60ceb3bdac4197fb7266d2df8155edacda.1759329363.git.mchehab+huawei@kernel.org>
When used in practice, one may want to have multiple header
files on a single rst file, like:
***********************
Digital TV uAPI symbols
***********************
.. contents:: Table of Contents
:depth: 2
:local:
Frontend
========
.. kernel-include:: include/uapi/linux/dvb/frontend.h
:generate-cross-refs:
:toc:
Demux
=====
.. kernel-include:: include/uapi/linux/dvb/dmx.h
:generate-cross-refs:
:toc:
...
So, don't add ..contents:: here.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <4bf353e5248133a3b0abd82519a38453402fe7c6.1759329363.git.mchehab+huawei@kernel.org>
On recent versions of openSUSE Tumbleweed, sphinx-buildis is no longer
a Python script, but something else. Such change is due to how
it now handles alternatives:
https://en.opensuse.org/openSUSE:Migrating_to_libalternatives_with_alts
The most common approach that distros use for alternatives is via
symlinks:
lrwxrwxrwx 1 root root 22 out 31 2024 /usr/bin/java -> /etc/alternatives/java
lrwxrwxrwx 1 root root 37 mar 5 2025 /etc/alternatives/java -> /usr/lib/jvm/java-21-openjdk/bin/java
With such approach, one can sun the script with either:
<sphinx>
python3 <script>
However, openSUSE's implementation uses an ELF binary (/usr/bin/alts),
which breaks the latter format.
It is needed to allow users to specify the Python version to be
used while building docs, as some distros like Leap 15.x are
shipped with:
- older, unsupported python3/python3-sphinx packages;
- more modern python3xx/python3xx-sphinx packages that work
properly.
On such distros, building docs require running make with:
make PYTHON3=python3.11 htmldocs
Heh, even on more moderen distros where python3-sphinx
is supported, one may still want to use a newer package,
for instance, due to performance issues, as:
- with Python < 3.11, kernel-doc is 3 times slower;
- while building htmldocs with Python 3.13/Sphinx 8.x
takes about 3 minutes on a modern machine, using
Sphinx < 8.0 can take up to 16 minutes to build docs
(7.x are the worse ones and require lots of RAM).
So, even with not too old distros, one still may want to use
for instance PYTHON3=python3.11.
To acommodate using PYTHON3 without breaking on Tumbleweed,
add a workaround that will only use:
$(PYTHON3) sphinx-build
if PYTHON3 env var is not default.
While here, drop the special check for venv, as, with venv,
we can just call sphinx-build directly without any extra
checks.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/all/883df949-0281-4a39-8745-bcdcce3a5594@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8917f862e0b8484c68408c274129c9f37a7aefb4.1758539031.git.mchehab+huawei@kernel.org>
The code here was meant to handle 3 functions:
1. allow having a separate conf.py file, per subdir;
2. generate a list of latex documents.
3. set "subproject" tag if SPHINXDIRS points to a subdir.
We don't have (1) anymore, and (3) is now properly handled
entirely inside conf.py.
So, only (3) is still needed, and this is a single-line change
at conf.py.
So, drop it, moving the remaining code to conf.py.
While here, drop a duplicated $(RUSTDOC) command-line argument.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ec998f9f268a401ca6aa36e3221d39c97efeccaa.1758361087.git.mchehab+huawei@kernel.org>
Before this patch, building htmldocs on opensuseLEAP works
fine:
# make htmldocs
Available Python versions:
/usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Using alabaster theme
Using Python kernel-doc
...
As the logic detects that Python 3.6 is too old and recommends
intalling python311-Sphinx. If installed, documentation builds
work like a charm.
Yet, some develpers complained that running python3.11 instead
of python3 should not happen. So, let's break the build to make
them happier:
$ make htmldocs
Python 3.6.15 not supported. Bailing out
You could run, instead:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs \
--sphinxdirs=. --conf=conf.py --builddir=Documentation/output --theme= --css= \
--paper=
Python 3.6.15 not supported. Bailing out
make[2]: *** [Documentation/Makefile:76: htmldocs] Error 1
make[1]: *** [Makefile:1806: htmldocs] Error 2
make: *** [Makefile:248: __sub-make] Error 2
It should be noticed that:
1. after this change, sphinx-pre-install needs to be called
by hand:
$ /usr/bin/python3.11 tools/docs/sphinx-pre-install
Detected OS: openSUSE Leap 15.6.
Sphinx version: 7.2.6
All optional dependencies are met.
Needed package dependencies are met.
2. sphinx-build-wrapper will auto-detect python3.11 and
suggest a way to build the docs using the parameters passed
via make variables. In this specific example:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs --sphinxdirs=. --conf=conf.py --theme= --css= --paper=
3. As this needs to be executed outside docs Makefile, it won't run
the validation check scripts nor build Rust documentation if
enabled, as the extra scripts are part of the docs Makefile.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <0635c311295300e9fb48c0ea607e2408910036e3.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Mauro Carvalho Chehab
Jonathan Corbet
Akira Yokosawa