99 Commits
Author SHA1 Message Date
Mauro Carvalho ChehabandJonathan Corbet ae63a5b920 docs: kdoc_re: get rid of NestedMatch class
Now that everything was converted to CMatch, we can get rid of
the previous NestedMatch implementation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <c82dd0d2c0ab330fc04925965091c448ccabb8fd.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet 600079fdcf docs: kdoc: replace NestedMatch with CMatch
Our previous approach to solve nested structs were to use
NestedMatch. It works well, but adding support to parse delimiters
is very complex.

Instead, use CMatch, which uses a C tokenizer, making the code more
reliable and simpler.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <900bff66f8093402999f9fe055fbfa3fa33a8d8b.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet 9aaeb817ef docs: c_lex: properly implement a sub() method for CMatch
Implement a sub() method to do what it is expected, parsing
backref arguments like \0, \1, \2, ...

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <dbc45b86db18783289d94cfdbba4b72792c47929.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet f1cf9f7cd6 docs: kdoc: create a CMatch to match nested C blocks
The NextMatch code is complex, and will become even more complex
if we add there support for arguments.

Now that we have a tokenizer, we can use a better solution,
easier to be understood.

Yet, to improve performance, it is better to make it use a
previously tokenized code, changing its ABI.

So, reimplement NextMatch using the CTokener class. Once it
is done, we can drop NestedMatch.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <fa818ea164216b17520b588e3f12b81499b76dd7.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet cd77a9aa20 docs: kdoc: use tokenizer to handle comments on structs
Better handle comments inside structs. After those changes,
all unittests now pass:

  test_private:
    TestPublicPrivate:
        test balanced_inner_private:                                 OK
        test balanced_non_greddy_private:                            OK
        test balanced_private:                                       OK
        test no private:                                             OK
        test unbalanced_inner_private:                               OK
        test unbalanced_private:                                     OK
        test unbalanced_struct_group_tagged_with_private:            OK
        test unbalanced_two_struct_group_tagged_first_with_private:  OK
        test unbalanced_without_end_of_line:                         OK

  Ran 9 tests

This also solves a bug when handling STRUCT_GROUP() with a private
comment on it:

	@@ -397134,7 +397134,7 @@ basic V4L2 device-level support.
	             unsigned int    max_len;
	             unsigned int    offset;
	             struct page_pool_params_slow  slow;
	-            STRUCT_GROUP( struct net_device *netdev;
	+            struct net_device *netdev;
	             unsigned int queue_idx;
	             unsigned int    flags;
	       };

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <f83ee9e8c38407eaab6ad10d4ccf155fb36683cc.1773074166.git.mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <054763260f7b5459ad0738ed906d7c358d640692.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet df50e848f6 docs: add a C tokenizer to be used by kernel-doc
Handling C code purely using regular expressions doesn't work well.

Add a C tokenizer to help doing it the right way.

The tokenizer was written using as basis the Python re documentation
tokenizer example from:
    https://docs.python.org/3/library/re.html#writing-a-tokenizer

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <39787bb8022e10c65df40c746077f7f66d07ffed.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet d5265f7af2 docs: kdoc: properly handle empty enum arguments
Depending on how the enum proto is written, a comma at the end
may incorrectly make kernel-doc parse an arg like " ".

Strip spaces before checking if arg is empty.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <4182bfb7e5f5b4bbaf05cee1bede691e56247eaf.1773074166.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <640784283d52c5fc52ea597344ecd567e2fb6e22.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet b1e64e30fc docs: kdoc: don't add broken comments inside prototypes
Parsing a file like drivers/scsi/isci/host.h, which contains
broken kernel-doc markups makes it create a prototype that contains
unmatched end comments.

That causes, for instance, struct sci_power_control to be shown this
this prototype:

    struct sci_power_control {
        * it is not. */ bool timer_started;
        */ struct sci_timer timer;
        * requesters field. */ u8 phys_waiting;
        */ u8 phys_granted_power;
        * mapped into requesters via struct sci_phy.phy_index */ struct isci_phy *requesters[SCI_MAX_PHYS];
    };

as comments won't start with "/*" anymore.

Fix the logic to detect such cases, and keep adding the comments
inside it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <18e577dbbd538dcc22945ff139fe3638344e14f0.1773074166.git.mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <12ac4a97e2bd5a19d6537122c10098690c38d2c7.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho ChehabandJonathan Corbet 023aabb6cc docs: python: add helpers to run unit tests
While python internal libraries have support for unit tests, its
output is not nice. Add a helper module to improve its output.

I wrote this module last year while testing some scripts I used
internally. The initial skeleton was generated with the help of
LLM tools, but it was higly modified to ensure that it will work
as I would expect.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <37999041f616ddef41e84cf2686c0264d1a51dc9.1773074166.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <d81be167b8cdeb003c1f8dcc7ad83a5ed2b520b6.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Ricardo UngererandJonathan Corbet 9b4e099c22 jobserver: Fix typo in docstring
This commit fixes small typos in the docstring of jobserver.py.

Signed-off-by: Ricardo Ungerer <ungerer.ricardo@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260316220218.568022-1-ungerer.ricardo@gmail.com>
2026-03-17 08:43:39 -06:00
Mauro Carvalho ChehabandJonathan Corbet ab9150972f docs: kdoc_output: better handle lists
On several functions, the return values are inside a bullet
list. Also, on some places, there are numbered lists as well.

Use a troff markup to format them, to avoid placing everything
on a single line.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <edea87623550a51086c23c9af0edc5e9fcce0ed6.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:39 -06:00
Mauro Carvalho ChehabandJonathan Corbet 908ae13b18 docs: kdoc_output: add support to handle code blocks
It is common to have code blocks inside kernel-doc markups.
By default, troff will group all lines altogether, producing a
very weird output. If a code block is detected by disabling
filling inside code blocks, re-enabling it afterwards.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <fb6f16ad345f7ec1b1ebe4c5ec7ea3d9cd6de4fb.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:39 -06:00
Mauro Carvalho ChehabandJonathan Corbet 4ec130cff6 docs: kdoc_output: add a logic to handle tables inside kernel-doc markups
specially when DOC is used, it is not uncommon to have tables
inside a kernel-doc markup.

Add support for simple tables and complex grid tables when output
in groff format.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <442ad76442c325044eb9f34a155d5f484341fb35.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet cde7c96f88 docs: kdoc_output: Change the logic to handle man highlight
The code inside ManFormat code to output man pages is too simple:
it produces very bad results when the content has tables or code
blocks.

Change the way lines are parsed there to allow adding extra
logic to handle some special cases.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6ae2301a40b3fcb4381dd9dda8c75d14f9616b46.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet e4dadcf510 docs: kdoc_output: pick a better default for modulename
Instead of placing the same data for modulename for all generated
man pages, use the directory from the filename used to produce
kernel docs as basis.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8a5d91c93c0b9b34c2f60e389f4464742804d0d6.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 26b4fdefc0 docs: kdoc_output: describe the class init parameters
As this class is part of the ABI used by both Sphinx kerneldoc
extension and docs/tools/kernel-doc, better describe what
parmeters are used to initialize ManOutput class.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <7c57f26150aae11fced259f30898a980b96efb68.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 4160533d05 docs: kdoc_output: fix naming for DOC markups
Right now, DOC markups aren't being handled properly, as it was
using the same name for all output.

Fix it by filling the title argument on a different way.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <11d809e5c4bec23240d3ace3f342dbb2a9263446.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 1a63342a27 docs: kdoc_output: don't use a different modulename for functions
It doesn't make much sense to have a different modulename just
for functions, but not for structs/enums/...

Use the same header everywere.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <978259bdf3e8d310c646ecf76ce56d054f6d5738.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 31938f120f docs: kdoc_output: use a single manual for everything
There's no reason why functions will be on a different manual.
Unify its name, calling it as "Kernel API Manual".

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <000e1174a551e97ad4710ad4f3750b22017bedd5.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 43874045fa docs: kdoc_output: remove extra attribute on man .TH headers
According with modern documents, groff .TH supports up to 5
arguments, but the logic passes 6. Drop the lastest one
("LINUX").

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <f5e480af877903b0596b6a56ef7a152eb8a10dbf.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet c1873e7743 docs: kdoc_output: use a method to emit the .TH header
All man emit functions need to add a .TH header. Move the code
to a common function, as we'll be addressing some issues at
the common code.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <2e55fcfe8724fde08a78635a1a3f8b449a6adf82.1772810752.git.mchehab+huawei@kernel.org>
2026-03-09 10:34:38 -06:00
Mauro Carvalho ChehabandJonathan Corbet 0d3ab0e4bb docs: kdoc_files: document KernelFiles() ABI
The KernelFiles is the main entry point to run kernel-doc,
being used by both tools/docs/kernel-doc and
Documentation/sphinx/kerneldoc.py.

It is also used on QEMU, which also uses the kernel-doc
libraries from tools/lib/python/kdoc.

Properly describe its ABI contract.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <13eb44c341232564eaf2a9c9de4782369fef57e1.1772810574.git.mchehab+huawei@kernel.org>
2026-03-09 10:23:20 -06:00
Mauro Carvalho ChehabandJonathan Corbet 861dcdb6ad docs: kdoc_files: allows the caller to use a different xforms class
While the main goal for kernel-doc is to be used inside the Linux
Kernel, other open source projects could benefit for it. That's
currently the case of QEMU, which has a fork, mainly due to two
reasons:

  - they need an extra C function transform rule;
  - they handle the html output a little bit different.

Add an extra optional argument to make easier for the code to be
shared, as, with that, QEMU can just create a new derivated class
that will contain its specific rulesets, and just copy the
remaining kernel-doc files as-is.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6b274ddbdcd9d438c6848e00e410a2f65ef80ec2.1772810574.git.mchehab+huawei@kernel.org>
2026-03-09 10:23:20 -06:00
Mauro Carvalho ChehabandJonathan Corbet 85c2a51357 docs: kdoc_parser: move nested match transforms to xforms_lists.py
As NestedMatch now has a sub method and a declaration close to
what KernRe does, we can move the rules to xforms_lists and
simplify kdoc_parser a little bit.

No functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <762ce2a58ff024c1b0b6f6a6e05020d1415b8308.1772469446.git.mchehab+huawei@kernel.org>
2026-03-03 10:47:25 -07:00
Mauro Carvalho ChehabandJonathan Corbet fc44c0a0b2 docs: kdoc_re: make NestedMatch use KernRe
Instead of using re_compile, let's create the class with the
regex and use KernRe to keep it cached.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <cdf900faf0ed8a08f8c6ac1db5a43342968c0739.1772469446.git.mchehab+huawei@kernel.org>
2026-03-03 10:47:25 -07:00