Commit Graph

60269 Commits

Author SHA1 Message Date
Luca Boccassi
b25e08a752 Merge pull request #22998 from DaanDeMeyer/journal-compact-split
journal: Add compact mode
2022-10-07 16:19:03 +02:00
Daan De Meyer
721620e8a3 journal: Add --convert= command to journalctl
--convert writes the journal files read by journalctl to the given
location. The location should be specified as a full journal file
path (e.g. /a/b/c/converted.journal). The directory specifies where
the converted journal files will be stored. The filename specifies
the naming convention the converted journal files will follow.
2022-10-07 12:28:09 +02:00
Daan De Meyer
e81710d3d0 journal: Store offsets to tail entry array objects in chain
Previously, we'd iterate an entry array from start to end every time
we added an entry offset to it. To speed up this operation, we cache
the last entry array in the chain and how many items it contains.
This allows the addition of an entry to the chain to be done in
constant time instead of linear time as we don't have to iterate
the entire chain anymore every time we add an entry.
2022-10-07 12:28:09 +02:00
Daan De Meyer
0e35afff1d journal: Introduce journal_file_data_payload()
journal_file_data_payload() retrieves the payload of a Data object,
optionally decompressing it and checking to see if matches a given
field. This function replaces all the decompression code in the sd-journal
codebase with a single function.

This commit should not introduce any changes in sd-journal behavior.
2022-10-07 12:28:05 +02:00
Daan De Meyer
a9089a6604 journal: Use 32-bit entry item object offsets in compact mode
To do this, we move EntryItem out of journal-def.h and turn it into
a host only struct in native endian mode so we can still use it to
ship the necessary info around.

Aside from that, the changes are pretty simple, we introduce some
extra functions to access the right field depending on the mode and
convert all the other code to use those functions instead of
accessing the raw fields.

We also drop the unused entry item hash field in compact mode. We
already stopped doing anything with this field a while ago, now we
actually drop it from the format in compact mode.
2022-10-07 12:20:57 +02:00
Daan De Meyer
99daf3ce03 journal: Use 32-bit entry array offsets in compact mode
Before:

OBJECT TYPE      ENTRIES SIZE
Unused           0       0B
Data             3610336 595.7M
Field            5310    285.2K
Entry            3498326 1.2G
Data Hash Table  29	 103.1M
Field Hash Table 29      151.3K
Entry Array      605991  1011.6M
Tag              0	 0B
Total            7720021 2.9G

After:

OBJECT TYPE      ENTRIES SIZE
Unused           0	 0B
Data             3562667 591.0M
Field            3971    213.6K
Entry            3498566 1.2G
Data Hash Table  20	 71.1M
Field Hash Table 20	 104.3K
Entry Array	 582647  505.0M
Tag              0	 0B
Total            7647891 2.4G
2022-10-07 12:20:08 +02:00
Daan De Meyer
d06727aec2 journal: Don't allocate objects above UINT32_MAX in compact mode
To allow storing offsets as 32-bit, we should never allocate objects
outside of the 32-bit range.
2022-10-07 12:20:08 +02:00
Daan De Meyer
c92f1ebe5d journal: Run unit tests with and without compact mode enabled 2022-10-07 12:16:11 +02:00
Yu Watanabe
02dfb6a3c9 Merge pull request #24930 from yuwata/network-drop-ndisc-vacuum
network: further follow-ups for recent NDisc PRs
2022-10-07 18:38:34 +09:00
Yu Watanabe
261979ed85 Merge pull request #24931 from bluca/news
hwdb and NEWS
2022-10-07 18:38:18 +09:00
Daan De Meyer
61297656c7 journal: Enable compact mode
We also add an environment variable $SYSTEMD_JOURNAL_COMPACT that
can be used to disable compact mode if needed (similar to
$SYSTEMD_JOURNAL_KEYED_HASH).
2022-10-07 11:28:12 +02:00
Daan De Meyer
87413812c9 journal: Add compact mode
This adds a new flag in preparation for incompatible journal changes
which will be gated behind this flag. The max file size of journal
files in compact mode is limited to 4 GiB.
2022-10-07 11:26:28 +02:00
Luca Boccassi
42c25d70f6 Update hwdb
ninja -C build update-hwdb
2022-10-07 11:00:28 +02:00
Luca Boccassi
23992ce1e9 NEWS: list contributors 2022-10-07 11:00:27 +02:00
Luca Boccassi
167420a394 NEWS: typos 2022-10-07 11:00:27 +02:00
Yu Watanabe
b3fac19f1b network: free timer event source for NDisc when link is freed
Though, it should be already freed already freed in link_stop_engines()
-> ndisc_stop(). Just for safety.
2022-10-07 14:32:13 +09:00
Yu Watanabe
44085d63f0 network: drop unnecessary call of ndisc_vacuum()
After the commit 773024685b, DNS servers
or domains are dropped when their lifefime become zero. Hence, it is not
necessary to try to them when writing state file.

Of course, because of the accuracy of the timer event source or priority
of event sources, a possibility is introduced that a DNS server or domain
with zero lifetime is stored in the state file. However, such entry will
be dropped soon when the timer event source is triggered. Hence, that
should not cause any real issues.
2022-10-07 14:17:28 +09:00
Zbigniew Jędrzejewski-Szmek
c76691d708 Merge pull request #24511 from martinetd/bpf1
libbpf: Add libbpf 1.0.0 compat
2022-10-06 19:01:33 +02:00
Arnaud Ferraris
5113436b05 repart: always honour --discard=no
Currently, even if `--discard=no` is passed to `systemd-repart`, the
`context_discard_gap_after()` function still runs normally, discarding
e.g. all blocks between the GPT and the start of the first partition.

This can lead to issues on some embedded devices, where this space
holds the bootloader and shouldn't be modified (creating a protective
partition there is not always possible due to the specifics of the boot
process of some ARM-based SoC's).

This commit ensures passing `--discard=no` would be enough to ensure
the bootloader isn't wiped in such cases.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@gmail.com>
2022-10-06 17:27:47 +02:00
Michal Koutný
3286770daa meson: Require TPM2 for measuring utilities
I happened to run build with openssl but no tpm2 and ran into issues
like:

        [313/1382] Compiling C object systemd-measure.p/src_boot_measure.c.o
        FAILED: systemd-measure.p/src_boot_measure.c.o
        cc -Isystemd-measure.p -I. -I.. -Isrc/basic -I../src/basic -Isrc/fundamental -I../src/fundamental -Isrc/systemd -I../src/systemd -I../src/libsystemd/sd-bus -I../src/libsystemd/sd-device -I../src/libsystemd/sd-event -I../src/libsystemd/sd-hwdb -I../src/libsystemd/sd-id128 -I../src/libsystemd/sd-journal -I../src/libsystemd/sd-netlink -I../src/libsystemd/sd-network -I../src/libsystemd/sd-resolve -Isrc/shared -I../src/shared -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -std=gnu11 -O0 -g -Wno-missing-field-initializers -Wno-unused-parameter -Wdate-time -Wendif-labels -Werror=format=2 -Werror=format-signedness -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -Werror=int-conversion -Werror=overflow -Werror=override-init -Werror=return-type -Werror=shift-count-overflow -Werror=shift-overflow=2 -Werror=undef -Wfloat-equal -Wimplicit-fallthrough=5 -Winit-self -Wlogical-op -Wmissing-include-dirs -Wmissing-noreturn -Wnested-externs -Wold-style-definition -Wpointer-arith -Wredundant-decls -Wshadow -Wstrict-aliasing=2 -Wstrict-prototypes -Wsuggest-attribute=noreturn -Wunused-function -Wwrite-strings -Wno-unused-result -Werror=missing-declarations -Werror=missing-prototypes -fdiagnostics-show-option -fno-common -fno-strict-aliasing -fstack-protector -fstack-protector-strong -fvisibility=hidden --param=ssp-buffer-size=4 -fno-omit-frame-pointer -Werror=shadow -include config.h -DOPENSSL_LOAD_CONF -MD -MQ systemd-measure.p/src_boot_measure.c.o -MF systemd-measure.p/src_boot_measure.c.o.d -o systemd-measure.p/src_boot_measure.c.o -c ../src/boot/measure.c
        ../src/boot/measure.c: In function ‘verb_sign’:
        ../src/boot/measure.c:710:48: error: variable ‘c’ has initializer but incomplete type
          710 |         _cleanup_(tpm2_context_destroy) struct tpm2_context c = {};

        [308/1382] Compiling C object systemd-pcrphase.p/src_boot_pcrphase.c.o
        FAILED: systemd-pcrphase.p/src_boot_pcrphase.c.o
        cc -Isystemd-pcrphase.p -I. -I.. -Isrc/basic -I../src/basic -Isrc/fundamental -I../src/fundamental -Isrc/systemd -I../src/systemd -I../src/libsystemd/sd-bus -I../src/libsystemd/sd-device -I../src/libsystemd/sd-event -I../src/libsystemd/sd-hwdb -I../src/libsystemd/sd-id128 -I../src/libsystemd/sd-journal -I../src/libsystemd/sd-netlink -I../src/libsystemd/sd-network -I../src/libsystemd/sd-resolve -Isrc/shared -I../src/shared -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -std=gnu11 -O0 -g -Wno-missing-field-initializers -Wno-unused-parameter -Wdate-time -Wendif-labels -Werror=format=2 -Werror=format-signedness -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -Werror=int-conversion -Werror=overflow -Werror=override-init -Werror=return-type -Werror=shift-count-overflow -Werror=shift-overflow=2 -Werror=undef -Wfloat-equal -Wimplicit-fallthrough=5 -Winit-self -Wlogical-op -Wmissing-include-dirs -Wmissing-noreturn -Wnested-externs -Wold-style-definition -Wpointer-arith -Wredundant-decls -Wshadow -Wstrict-aliasing=2 -Wstrict-prototypes -Wsuggest-attribute=noreturn -Wunused-function -Wwrite-strings -Wno-unused-result -Werror=missing-declarations -Werror=missing-prototypes -fdiagnostics-show-option -fno-common -fno-strict-aliasing -fstack-protector -fstack-protector-strong -fvisibility=hidden --param=ssp-buffer-size=4 -fno-omit-frame-pointer -Werror=shadow -include config.h -DOPENSSL_LOAD_CONF -MD -MQ systemd-pcrphase.p/src_boot_pcrphase.c.o -MF systemd-pcrphase.p/src_boot_pcrphase.c.o.d -o systemd-pcrphase.p/src_boot_pcrphase.c.o -c ../src/boot/pcrphase.c
        ../src/boot/pcrphase.c: In function ‘determine_banks’:
        ../src/boot/pcrphase.c:117:24: error: unknown type name ‘TPMI_ALG_HASH’
          117 |         _cleanup_free_ TPMI_ALG_HASH *algs = NULL;

Guarding the utilites with HAVE_TPM2 fixes the issue for me.

This complements #24811.
2022-10-06 22:57:45 +09:00
Dominique Martinet
87e462f713 libbpf: add compat helpers for libbpf down to 0.1.0
- new symbols are available from libbpf 0.6.0 so could be used with
libbpf.so.0, but we're sure the old symbols will be there and this
simplifies code
- detection at runtime should always work, regardless of whether systemd
has been compiled with older or newer libbpf and runs with older or newer
libbpf
2022-10-06 21:33:55 +09:00
drosdeck
cbf8fad440 Fix key toggle touchpad and programmable buttom for Positivo N14 2022-10-06 13:50:44 +09:00
Aleksey Vasenev
2be1ae54ba ata_id: Fixed getting Response Code from SCSI Sense Data (#24921)
The Response Code is contained in the first byte of the SCSI Sense Data.
Bit number 7 is reserved or has a different meaning for some Response Codes
and is set to 1 for some drives.
2022-10-06 13:50:30 +09:00
Daan De Meyer
5a967e2887 Try to load libbpf.so.1 as well
libbpf had a soname bump. Our usage of libbpf is compatible with
both libbpf.so.0 and libbpf.so.1, so let's try to load from both.
2022-10-06 07:31:23 +09:00
Daan De Meyer
6b8085db68 libbpf: Remove use of deprecated APIs 2022-10-06 07:31:23 +09:00