Commit Graph

227 Commits

Author SHA1 Message Date
Evgeny Vereshchagin
2fd1beb3e2 oss-fuzz: move apt-gets and pips to the systemd repository
to be able to control our dependencies right here without
sending PRs like https://github.com/google/oss-fuzz/pull/5199 and
https://github.com/google/oss-fuzz/pull/5601.

It should also allow us to pin meson to let Dependabot keep track of
it and jump from one version to another without breaking anything
2021-11-19 08:52:28 +00:00
Christian Brauner
a6d1760024 build: preserve correct mode when generating files via jinja2
When using "capture : true" in custom_target()s the mode of the source
file is not preserved when the generated file is not installed and so
needs to be tweaked manually. Switch from output capture to creating the
target file and copy the permissions from the input file.

Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
2021-11-08 12:06:48 +00:00
Zbigniew Jędrzejewski-Szmek
5f035b13de meson-render-jinja2: use ast.literal_eval()
Imports are sorted in the usual fashion: stdlib first.

literal_eval() parses string/numbers/lists/sets/dicts, and nothing else, while
eval will execute any python code. Using literal_eval() is generally more
correct, because it avoids the risk of side effects from the parsed expression.
In this case, we generate the parsed strings ourselves, so it's very unlikely
to have anything unexpected in the expressions. But let's do the correct thing
anyway.
2021-11-05 14:56:32 +01:00
Luca Boccassi
948def4af6 tree-wide: fix SPDX short identifier for LGPL-2.1-or-later
https://spdx.dev/ids/#:~:text=Allowing%20later%20versions%20of%20a%20license
https://spdx.org/licenses/LGPL-2.1-or-later.html
2021-10-01 17:27:34 +01:00
Zbigniew Jędrzejewski-Szmek
090ade7ee7 licensing: add spdx header to chromiumos helper, move license file
It makes it easier to process the license automatically like other files.
The text of the license in tools/chromiumos/LICENSE matches
https://spdx.org/licenses/BSD-3-Clause.html exactly.
2021-10-01 14:45:00 +02:00
Frantisek Sumsal
f7e0d22d76 tools: shellcheck-ify tool scripts 2021-09-30 12:27:06 +02:00
Luca Boccassi
95b63c755b git-contrib: copypaste-friendly output
Format output in a manner that can be copypasted as-is to NEWS.
That is, with 8 spaces indentation and wrapped at 80 columns.

Before:

$ tools/git-contrib.sh
        Ben Stockett,
        Carl Lei,
        Frantisek Sumsal,
        Gibeom Gwon,
        Hugo Osvaldo Barrera,
        James Hilliard,
        Jan Palus,
        Lennart Poettering,
        Luca Boccassi,
        Luca BRUNO,
        Mike Gilbert,
        nassir90,
        nl6720,
        Raul Tambre,
        Yegor Alexeyev,
        Yu Watanabe,
        Zbigniew Jędrzejewski-Szmek,

After:

        Contributions from: Ben Stockett, Carl Lei, Frantisek Sumsal,
        Gibeom Gwon, Hugo Osvaldo Barrera, James Hilliard, Jan Palus,
        Lennart Poettering, Luca Boccassi, Luca BRUNO, Mike Gilbert,
        nassir90, nl6720, Raul Tambre, Yegor Alexeyev, Yu Watanabe,
        Zbigniew Jędrzejewski-Szmek
2021-07-19 15:39:26 +09:00
Zbigniew Jędrzejewski-Szmek
d080734dcb man: add "DNS resource record types" section 2021-06-29 10:44:18 +02:00
Zbigniew Jędrzejewski-Szmek
19c79550b4 tools/analyze-dump-sort: a helper to compare two 'systemd-analyze dump' outputs
Lines in the dumps are ordered by some pseudo-random hashmap entry order, which
makes it hard to diff two outputs. This sort the entries alphabetically, and
also sorts items within the entries, and supresses timestamps and other fields
which always vary.

We could sort the output inside of systemd itself, but it'd make things more
complex, and we probably don't need output to be sorted in most cases. It also
wouldn't be enough, because timestamps and such would still need to be ignored
to do a nice diff. So I think doing the sorting and suppression in a python
helper is a better approach.
2021-06-04 12:09:58 +02:00
Luca Boccassi
c5fd89adcd core: make libbpf a dlopen() dependency 2021-05-25 12:59:26 +01:00
Zbigniew Jędrzejewski-Szmek
44ff8df777 Drop dependency on m4
m4 was hugely popular in the past, because autotools, automake, flex, bison and
many other things used it. But nowadays it much less popular, and might not even
be installed in the buildroot. (m4 is small, so it doesn't make a big difference.)

(FWIW, Fedora dropped make from the buildroot now,
https://fedoraproject.org/wiki/Changes/Remove_make_from_BuildRoot. I think it's
reasonable to assume that m4 will be dropped at some point too.)

The main reason to drop m4 is that the syntax is not very nice, and we should
minimize the number of different syntaxes that we use. We still have two
(configure_file() with @FOO@ and jinja2 templates with {{foo}} and the
pythonesque conditional expressions), but at least we don't need m4 (with
m4_dnl and `quotes').
2021-05-19 10:24:43 +09:00
Zbigniew Jędrzejewski-Szmek
6b1aac3ccf meson: replace some m4 templates with jinja2
m4 was nice in '85, but the syntax feels a bit dated. Since we use python for
meson, let's use a popular python templating engine to replace some m4 usage.

A little nicety is that typos are caught:

FAILED: sysusers.d/systemd-remote.conf
/usr/bin/meson --internal exe --capture sysusers.d/systemd-remote.conf -- /home/zbyszek/src/systemd/tools/meson-render-jinja2.py config.h ../sysusers.d/systemd-remote.conf.j2
Traceback (most recent call last):
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 28, in <module>
    print(render(sys.argv[2], defines))
  File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 24, in render
    return template.render(defines)
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 8, in top-level template code
jinja2.exceptions.UndefinedError: 'HAVE_MICROHTTP' is undefined

This checking mirrors what 349cc4a507 did for C defines.
2021-05-19 10:24:43 +09:00
Jörg Thalheim
f96bc66901 fix bash shebangs 2021-05-17 22:27:41 +01:00
Zbigniew Jędrzejewski-Szmek
b8f1045fe7 Revert "tools/make-directive-index: parallelize"
This reverts commit a2031de849.

The patch itself seems OK, but it exposes a bug in lxml or libxml2-2.9.12 which
was just released. This is being resolved in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/255, but it might be while. So
let's revert this for now to unbreak our CI.

Fixes #19601.
2021-05-14 23:16:24 +01:00
Zbigniew Jędrzejewski-Szmek
a2031de849 tools/make-directive-index: parallelize
I occasionally do 'build/man/man systemd.directives' when working on man pages,
and it's annoying slow. By paralellizing the parsing of xml, we can make it a
bit faster.

This is still rather innefficient. Only the parsing part is serialized, xml is
still produced serially at the end, which is hard to avoid.

$ ninja -C build man/systemd.directives.xml
before:
8.20s user 0.21s system 99% cpu 8.460 total
8.33s user 0.18s system 98% cpu 8.619 total
8.72s user 0.19s system 98% cpu 9.019 total

after:
13.99s user 0.73s system 345% cpu 4.262 total
14.15s user 0.35s system 348% cpu 4.161 total
14.33s user 0.35s system 339% cpu 4.321 total

I.e. it uses almost twice as much cpu, but cuts the wallclock time down (on a
2-core/4-thread cpu) to about half too, which is an overall win if you're just
trying to render the man page.

The change from list and .append() to set and .add() is something that could
have been done before too, but it's noticable now. It cuts down on the
serialization/deserialization time (about .2s).
2021-05-13 11:53:01 +02:00
Julia Kartseva
cf4f9a57f2 bpf: add build script for bpf programs
Add a build script to compile bpf source code. A program in restricted
C is compiled into an object file. Object file is converted to BPF
skeleton [0] header file.
If build with custom meson build rule, the target header will reside in
build/ directory (not in source tree), e.g the path for socket_bind:
`build/src/core/bpf/socket_bind/socket-bind.skel.h`

Script runs the phases:
* clang to generate *.o from restricted C
* llvm-strip to remove useless DWARF info
* bpf skeleton generation with bpftool
These phases are logged to stderr for debug purposes.

To include BTF debug information, -g option is passed to clang.

[0] https://lwn.net/Articles/806911/
2021-04-26 16:07:41 -07:00
Frantisek Sumsal
3b6fd3c1de tools: shellcheck-ify most of the tool scripts 2021-04-20 20:11:13 +02:00
Frantisek Sumsal
34fde9f898 test: check if the unit file fuzzer corpora is up to date
This follows a similar pattern we already have in place for
networkd-related directives.
2021-04-05 14:41:05 +01:00
Zbigniew Jędrzejewski-Szmek
b242d2dec9 git-contrib: use non-breaking spaces in names
Some people have initials or abbreviated parts in the name and looks strange
when a line break occurs in the middle. Let's keep each name in one line.
2021-03-30 13:17:59 +02:00
Lennart Poettering
1ab4edae12 tools: exclude Weblate bot from "ninja git-contrib"
Bots are not people, no reason to say thanks to them and list them as
authors or contributors.
2021-03-17 16:53:14 +01:00
Zbigniew Jędrzejewski-Szmek
a3fbf67180 hwdb: update for v248
Seems to be additions and corrections as usual.
Includes an update of the chromiumos autosuspend rules.
2021-03-08 14:27:48 +01:00
Zbigniew Jędrzejewski-Szmek
64eb60b8c5 update-dbus-docs: use color in summary 2021-02-06 11:41:42 +01:00
Zbigniew Jędrzejewski-Szmek
7bd5b8614d update-dbus-docs: say "MODIFIED" not "OUTDATED"
When executed in test mode, "OUTDATED" is appropriate. But when executed
to actually update the text, after the tool executes, those pages are the
opposite, not outdated.
2021-02-06 11:41:42 +01:00
Evgeny Vereshchagin
77591e9732 oss-fuzz: show meson logs
It should help to make it more clear what causes issues like
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=30140
and https://github.com/google/oss-fuzz/pull/5084
2021-02-01 22:51:43 +09:00
Anita Zhang
934d0d023e tools: make update-dbus-docs compatible with Python 3.6
668b3a42fe allowed update-dbus-docs.py to start
running on Cent OS 8 (instead of skipping). But subprocess.check_output()'s
text argument didn't exist until Python 3.7 and C8 is still running
Python 3.6. Use universal_newlines instead for backwards compatibility.
2021-02-01 11:21:10 +00:00