Commit Graph

63 Commits

Author SHA1 Message Date
Vegard Nossum
c160f2ac85 scripts/get_abi: fix source path leak
commit 5889d6ede53bc17252f79c142387e007224aa554 upstream.

The code currently leaks the absolute path of the ABI files into the
rendered documentation.

There exists code to prevent this, but it is not effective when an
absolute path is passed, which it is when $srctree is used.

I consider this to be a minimal, stop-gap fix; a better fix would strip
off the actual prefix instead of hacking it off with a regex.

Link: https://mastodon.social/@vegard/111677490643495163
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20231231235959.3342928-1-vegard.nossum@oracle.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-01-31 16:17:01 -08:00
SeongJae Park
5b5bfecaa3 scripts/get_abi: Fix wrong script file name in the help message
The help message of 'get_abi.pl' is mistakenly saying it's
'abi_book.pl'.  This commit fixes the wrong name in the help message.

Fixes: bbc249f2b8 ("scripts: add an script to parse the ABI files")
Signed-off-by: SeongJae Park <sj@kernel.org>
Link: https://lore.kernel.org/r/20220419121636.290407-1-sj@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-04-24 10:38:44 +02:00
Linus Torvalds
b4a5ea09b2 Merge tag 'docs-5.18-2' of git://git.lwn.net/linux
Pull more documentation updates from Jonathan Corbet:
 "Some late-arriving documentation improvements.

  This is mostly build-system fixes from Mauro and Akira; I also took
  the liberty of dropping in my 'messy diffstat' document"

* tag 'docs-5.18-2' of git://git.lwn.net/linux:
  docs: Add a document on how to fix a messy diffstat
  docs: sphinx/requirements: Limit jinja2<3.1
  Documentation: kunit: Fix cross-referencing warnings
  scripts/kernel-doc: change the line number meta info
  scripts/get_abi: change the file/line number meta info
  docs: kernel_include.py: add sphinx build dependencies
  docs: kernel_abi.py: add sphinx build dependencies
  docs: kernel_feat.py: add build dependencies
  scripts/get_feat.pl: allow output the parsed file names
  docs: kfigure.py: Don't warn of missing PDF converter in 'make htmldocs'
  Documentation: Fix duplicate statement about raw_spinlock_t type
2022-03-31 12:10:42 -07:00
Mauro Carvalho Chehab
92b6de17b2 scripts/get_abi: change the file/line number meta info
In order to make it more standard and ReST compatible,
change the meta-tag used with --enable-lineno from:

	#define LINENO

to
	.. LINENO

In practice, no functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Link: https://lore.kernel.org/r/125ffd31fbc77ad9eee4d6906e1830b8162fa6ca.1648290305.git.mchehab@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2022-03-28 13:53:46 -06:00
Michal Simek
5bff9632b5 scripts: get_abi.pl: Fix typo in help message
Fix misspelled word in the script. (s/COMAND/COMMAND/).

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Link: https://lore.kernel.org/r/1ff3605db3dead41bbde33fbbff6754900eceead.1645789205.git.michal.simek@xilinx.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-03-18 14:27:07 +01:00
Jonathan Neuschäfer
aa21a1bf97 scripts/get_abi.pl: Ignore hidden files
get_abi.pl currently collects every file in Documentation/ABI. This
causes a UnicodeDecodeError in Documentation/sphinx/kernel_abi.py,
when it finds my Vim swap files (.foo.swp) in the directory.

To avoid such issues, ignore hidden files in get_abi.pl.

Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Link: https://lore.kernel.org/r/20220129005019.2090996-1-j.neuschaefer@gmx.net
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-02-04 16:43:09 +01:00
Mauro Carvalho Chehab
df2205de92 scripts: get_abi.pl: better generate regex from what fields
Using repeating sequencies of .* seem to slow down the
processing speed on some cases. Also, currently, a "."
character is not properly handled as such.

Change the way regexes are created, in order to produce
better search expressions.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/c69c01c12b1b30466177dcb17e45f833fb47713d.1632994565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-10-05 16:23:38 +02:00
Mauro Carvalho Chehab
4dcce5b081 scripts: get_abi.pl: fix fallback rule for undefined symbols
The rule that falls back to the long regex list is wrong:
it is just running again the same loop it did before.

change it to look at the "others" table.

That slows the processing speed, but provides a better
list of undefined symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a3ba919e9a9208a5f012a13c9674c362a9d73169.1632994565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-10-05 16:23:38 +02:00
Mauro Carvalho Chehab
e5c044c8a9 scripts: get_abi.pl: make undefined search more deterministic
Sort keys on hashes during undefined search, in order to
make the script more deterministic.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5dc55fd42e632a24a48f95212aa6c6bc4b2d11fd.1632865873.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-29 09:10:52 +02:00
Mauro Carvalho Chehab
28331a011d scripts: get_abi.pl: show progress
As parsing the sysfs entries can take a long time, add
progress information.

The progress logic will update the stats on every second,
or on 1% steps of the progress.

When STDERR is a console, it will use a single line, using
a VT-100 command to erase the line before rewriting it.
Otherwise, it will put one message on a separate line.
That would help to identify what parts of sysfs checking
that it is taking more time to process.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/4e581dcbec21ad8a60fff883498018f96f13dd1c.1632823172.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:42 +02:00
Mauro Carvalho Chehab
2833e30aa0 scripts: get_abi.pl: use STDERR for search-string and show-hints
On undefined checks, use STDOUT only for the not found entries.

All other data (search-string and show-hints) is printed at
STDERR.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/51c6a39c82f73b441030c51bf905a1f382452a67.1632823172.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:42 +02:00
Mauro Carvalho Chehab
42f09848cf scripts: get_abi.pl: update its documentation
The current highlight schema is not working properly. So, use,
instead, Pod::Text.

While here, also update the copyright in order to reflect the latest
changes and the e-mail I'm currently using.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/89fcd301e065ed86dfd8670725144b196266b6a4.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:36 +02:00
Mauro Carvalho Chehab
87b58c6fae scripts: get_abi.pl: fix parse logic for DT firmware
It doesn't make any sense to parse ABI entries under
/sys/firmware, as those are either specified by ACPI specs
or by Documentation/devicetree.

The current logic to ignore firmware entries is incomplete,
as it ignores just the relative name of the file, and not
its absolute name. This cause errors while parsing the
symlinks.

So, rewrite the logic for it to do a better job.

Tested with both x86 and arm64 (HiKey970) systems.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/1c806eaec96f6706db4b041bbe6a0e2519e9637e.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:36 +02:00
Mauro Carvalho Chehab
3a1cc06c0e scripts: get_abi.pl: produce an error if the ref tree is broken
The logic under graph_add_file should create, for every entry, a
__name name array for all entries of the tree. If this fails, the
symlink parsing will break.

Add an error if this ever happens.

While here, improve the output of data dumper to be more
compact and to avoid displaying things like $VAR1=.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/e7dd4d70e206723455d50c851802c8bb6c34941d.1632750315.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:36 +02:00
Mauro Carvalho Chehab
ff3777d0d6 scripts: get_abi.pl: create a valid ReST with duplicated tags
As warned, /sys/bus/iio/devices/iio:deviceX/fault_ovuv is defined 2 times:

	Warning: /sys/bus/iio/devices/iio:deviceX/fault_ovuv is defined 2 times:  ./Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:14  ./Documentation/ABI/testing/sysfs-bus-iio-temperature-max31865:0

The logic with joins the two entries is just places the paragraph
for the second entry after the previous one. That could cause more
warnings, as the produced ReST may become invalid, as in the case of
this specific symbol, which ends with a table:

	/new_devel/v4l/docs/Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:2: WARNING: Malformed table.
	No bottom table border found or no blank line after table bottom.

	===  =======================================================
	'1'  The input voltage is negative or greater than VDD.
	'0'  The input voltage is positive and less than VDD (normal
	     state).
	===  =======================================================
	/new_devel/v4l/docs/Documentation/ABI/testing/sysfs-bus-iio-temperature-max31856:2: WARNING: Blank line required after table.

Address it by adding two blank lines before joining duplicated
symbols.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/4ad2e3a65f781f0f8d40bb75aa5a07aca80564d6.1632740376.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-28 12:45:21 +02:00
Mauro Carvalho Chehab
d4771993f2 scripts: get_abi.pl: ensure that "others" regex will be parsed
The way the search algorithm works is that reduces the number of regex
expressions that will be checked for a given file entry at sysfs. It
does that by looking at the devnode name. For instance, when it checks for
this file:

	/sys/bus/pci/drivers/iosf_mbi_pci/bind

The logic will seek only the "What:" expressions that end with "bind".
Currently, there are just a couple of What expressions that matches
it:

	What: /sys/bus/fsl\-mc/drivers/.*/bind
	What: /sys/bus/pci/drivers/.*/bind

It will then run an O(n²) algorithm to seek, which runs quickly
when there are few regexs to seek. There are, however, some What:
expressions that end with a wildcard. Those are harder to process.
Right now, they're all grouped together at the "others" group.

As those don't depend on the basename of the node, add an extra
loop to ensure that those will be processed at the end, if
not done yet.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/9fe7ab46f67575def5db9e83034e9fab43846d84.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
f34f67292b scripts: get_abi.pl: precompile what match regexes
In order to earn some time during matches, pre-compile regexes.

Before this patch:
	$ time ./scripts/get_abi.pl undefined |wc -l
	6970

	real	0m54,751s
	user	0m54,022s
	sys	0m0,592s

Afterwards:

	$ time ./scripts/get_abi.pl undefined |wc -l
	6970

	real	0m5,888s
	user	0m5,310s
	sys	0m0,562s

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/ec45de8fcae791aab0880644974a110424423e68.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
cb06b8ddeb scripts: get_abi.pl: stop check loop earlier when regex is found
Right now, there are two loops used to seek for a regex. Make
sure that both will be skip when a match is found.

While here, drop the unused $defined variable.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/2ba722d2cdbe7c7d0f1d1b58d350052576d1d703.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
0cd9e25b08 scripts: get_abi.pl: ignore some sysfs nodes earlier
When checking for undefined symbols, some nodes aren't easy
or don't make sense to be checked right now. Prevent allocating
memory for those, as they'll be ignored anyway.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/5228789cbef8241d44504ad29fca5cab356cdc53.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
9263589422 scripts: get_abi.pl: Better handle leaves with wildcards
When the the leaf of a regex ends with a wildcard, the speedup
algorithm to reduce the number of regexes to seek won't work.

So, when those are found, place at the "others" exception.

That slows down the search from 0.14s to 1 minute on my
machine, but the results are a lot more consistent.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/60bb97cf337333783f9f52e114b896439e9cc215.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
46f661fd0f scripts: get_abi.pl: improve debug logic
Add a level for debug, in order to allow it to be extended to
debug other parts of the script.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/0203416c6c418abb4fc20577a5f48d0d2a41bae7.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
45495db979 scripts: get_abi.pl: call get_leave() a little late
The $what conversions need to replace some characters to avoid
breaking regex expressions found on some What:.

only after replacing them back, the script should get the
$leave devnode.

Fixes: ca8e055c22 ("scripts: get_abi.pl: add a graph to speedup the undefined algorithm")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/a21631f8a884f50a962beafdd800f27891348d95.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
e27c42a52e scripts: get_abi.pl: Fix get_abi.pl search output
Currently, the get_abi.pl will print an invalid symbol
(\xac character). Fix it.

Fixes: ab9c14805b ("scripts: get_abi.pl: Better handle multiple What parameters")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/fb27ac372e38f5ae9d088f9f4e9710c659e0b9e8.1632411447.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-23 18:45:08 +02:00
Mauro Carvalho Chehab
ca8e055c22 scripts: get_abi.pl: add a graph to speedup the undefined algorithm
Searching for symlinks is an expensive operation with the current
logic, as it is at the order of O(n^3). In practice, running the
check spends 2-3 minutes to check all symbols.

Fix it by storing the directory tree into a graph, and using
a Breadth First Search (BFS) to find the links for each sysfs node.

With such improvement, it can now report issues with ~11 seconds
on my machine.

It comes with a price, though: there are more symbols reported
as undefined after this change. I suspect it is due to some
sysfs circular loops that are dropped by BFS. Despite such
increase, it seems that the reports are now more coherent.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/f5c1e7b14a27132821c08f0459ba9aea3ed69028.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-21 18:32:53 +02:00
Mauro Carvalho Chehab
0b87a1b81b scripts: get_abi.pl: Ignore fs/cgroup sysfs nodes earlier
In order to speedup the parser and store less data, handle
fs/cgroup exceptions a lot earlier.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/caa37831c9e02ae58677d1515ed7cee94f52ea9d.1631957565.git.mchehab+huawei@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-09-21 18:32:53 +02:00