* features:
s390/configs: Increase CONFIG_LOCKDEP_CHAINS_BITS
s390: Add support for DCACHE_WORD_ACCESS (again)
s390/mm: Use goto statement in do_secure_storage_access()
s390/mm: Use handle_fault_error() in do_secure_storage_access()
s390/mm: Remove folio handling for kernel faults in do_secure_storage_access()
s390/mm: Fix handling of vmalloc area in do_secure_storage_access()
s390/mm: Use lock_mm_and_find_vma() in do_secure_storage_access()
s390/mm: Add missing mm check to do_secure_storage_access()
KVM: s390: pv: Use VM_SPARSE area for guest variable storage area
s390/spinlock: Add contention tracepoints to lock slowpath
s390/ipl: Improve readability
s390/ipl: Use ARRAY_SIZE macro
s390/maccess: Use proper PTE accessors for copying old memory
s390/pkey: Rework ioctl functions error paths
s390/ap: Use mutex_lock_killable() in ap_bus_force_rescan()
s390/vdso: Use symbolic constants for the PHDR permission flags
s390/vdso: Pass --eh-frame-hdr to the linker
s390/syscalls: Use define instead of '1' to indicate PER trap
s390/traps: Remove PIF_GUEST_FAULT
s390/uapi: Remove obsolete unistd_32.h from Kbuild file
s390: Select SWIOTLB_DYNAMIC and DMA_COHERENT_POOL
s390/pv: Enable SWIOTLB_ANY for s390 PV
s390/traps: Add exception statistics
s390/configs: Enable cpuidle driver on s390
s390/idle: Introduce cpuidle for s390
s390: Enable TIF_POLLING_NRFLAG
tick: Remove arch_needs_cpu
s390/tick: Remove CIF_NOHZ_DELAY flag
s390/mm: Use set_pmd() / set_pud() for hugetlb pagetable entries
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The hrtimer_sleeper structure's ->task field is used as a flag to indicate
that the associated hrtimer has expired. This means that the hrtimer
handler can be storing to this field while other code is loading from it
to check for expiry. Note that additional races appear for hrtimers that
can be restarted, which could be argued to be a user error. However, that
is no reason to let the compiler introduce additional confusion, and to
this end, the hrtimer_sleeper_task_get() was introduced, use of which also
has the benefit of avoiding open-code access to hrtimer_sleeper innards.
Therefore, apply this accessor to schedule_hrtimeout_range_clock().
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Anna-Maria Behnsen <anna-maria@linutronix.de>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The hrtimer_sleeper structure's ->task field is used as a flag to indicate
that the associated hrtimer has expired. This means that the hrtimer
handler can be storing to this field while other code is loading from it
to check for expiry. Note that additional races appear for hrtimers that
can be restarted, which could be argued to be a user error. However,
that is no reason to let the compiler introduce additional confusion.
Therefore, mark data-racy accesses to the hrtimer_sleeper ->task field
using READ_ONCE() (using a new hrtimer_sleeper_task_get() access function)
and WRITE_ONCE() (using a new hrtimer_sleeper_task_set() access function).
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Anna-Maria Behnsen <anna-maria@linutronix.de>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Thomas Gleixner <tglx@kernel.org>
Running kernel-doc on timer_migration.h reports:
Warning: kernel/time/timer_migration.h:105 bad line:
The empty line separating the @available paragraph from the @idle
member documentation in the struct tmigr_cpu kernel-doc block lacks
the " *" line prefix that kernel-doc requires on every line inside a
block. The header is not scanned by the build-time kernel-doc checks,
so the warning only shows up when kernel-doc is invoked on the file
directly.
Add the missing prefix. The empty line was introduced when the
@available documentation was expanded by commit 45a13ba52c
("timers/migration: Update stale @online doc to @available").
No functional change.
Signed-off-by: Babanpreet Singh <bbnpreetsingh@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Claude:claude-sonnet-5
Link: https://patch.msgid.link/20260717035947.7-1-bbnpreetsingh@gmail.com
Now that nothing inflates tick_length beyond tick_length_base (the
adjtime path was converted to use time_offset in the previous commit),
the two fields are always equal.
Remove tick_length_base and keep tick_length as the single field.
Remove the per-second reset and the delta update in
ntp_update_frequency() since there is no separate base to track.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.6-1m
Link: https://patch.msgid.link/20260621220051.1030462-8-dwmw2@infradead.org
time_offset (the exponential PLL phase slew) and time_adjust (the
linear adjtime() slew) can be asked to move the clock in opposite
directions. second_overflow() folds only their *net* into the per-tick
skew_delta, so the cancelling overlap would never be drained from
either tracker by the per-tick code — and if they cancel exactly,
skew_delta is zero and neither converges at all.
Arguably we could just let one of them entirely cancel out the other
immediately, but that would be a change in userspace-visible behaviour.
Instead, preserve the existing behaviour by calculating the "conflict"
portion between the opposing skew each second, and transferring that
amount directly from one tracker to the other.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-7-dwmw2@infradead.org
The legacy adjtime() slew (ADJ_OFFSET_SINGLESHOT) was the last user of
tick_length != tick_length_base: it slewed the clock by inflating
tick_length directly, which delivered the correction imprecisely (e.g.
delivering only 4997.5µs when asked for a 5ms skew).
Deliver it accurately through the same per-tick mechanism that is now
used for time_offset, allowing it to contribute to skew_delta and thus
drive the delivery through ntp_error and mult selection.
To allow for accurate accounting, store the sub-microsecond part of
time_adjust is separately, while keeping time_adjust in microseconds
as that's the external API.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-6-dwmw2@infradead.org
Currently, the phase offset of time_offset and time_adjust is delivered
by adjusting tick_length in second_overflow(), and immediately draining
time_offset/time_adjust by the amount that the tick_length adjustment is
*estimated* to cause. This is fairly approximate, in part because it is
not always correct to assume that precisely NTP_INTERVAL_FREQ ticks will
occur between one call to second_overflow() and the next. It could also
over and under-run in the final second of delivery.
Instead of inflating tick_length, transfer the intended skew directly
into ntp_error each tick to achieve the desired rate.
In second_overflow(), calculate skew_delta which is the per-tick slew
rate, in the same units as time_offset: (ns << NTP_SCALE_SHIFT) / HZ.
In logarithmic_accumulation(), drain up to 'skew_delta' time units from
time_offset into ntp_error to drive the overall effective rate. The new
ntp_drain_skew() function returns the amount which is actually 'claimed'
by time_offset (and in a future patch, time_adjust). Any overrun which
is delivered by the changed 'mult' (as described below) but not claimed
by ntp_drain_skew() will remain in ntp_error to be corrected away in
subsequent ticks.
Simply transferring the precise amount from time_offset to ntp_error
would be sufficent to make the time *eventually* converge, however the
skew delivered is limited by the choice of { mult, mult+1 } each tick
and thus the convergence would be extremely slow.
In theory we could inflate ntp_err_mult with the magnitude of ntp_error
in the general case — but that would cause overcorrection in a tickless
kernel. Instead, in timekeeping_adjust(), take skew_delta into account
when calculating 'mult', such that the available {mult, mult+1} choices
bracket the overall effective rate *including* the skew, to avoid the
delta just building up in ntp_error.
The effect is that the inflated 'mult' causes ntp_error to grow because
xtime_interval is (e.g.) longer than the true tick_length. But then the
same delta is removed again as it's drained from time_offset.
This gives behaviour equivalent to the old tick_length += delta approach
but with exact per-tick accounting of the time_offset actually imparted
to the clock, and no overrun.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Link: https://patch.msgid.link/20260621220051.1030462-5-dwmw2@infradead.org
cycle_interval is an integer number of counter cycles per NTP interval,
so the real time it represents differs from the nominal
NTP_INTERVAL_LENGTH by up to half a counter period. For coarse
clocksources this is significant: the 3.579545 MHz ACPI PM timer at
HZ=1000 rounds 3579.545 cycles up to 3580, making each tick 1.000127 ms
(+127 PPM).
Commit a386b5af8e ("time: Compensate for rounding on odd-frequency
clocksources") introduced xtime_remainder to compensate for exactly
this, citing the same 127 PPM ACPI PM example. The compensation is
correct and necessary, but it was applied inside the timekeeping
accumulation in timekeeping.c: subtracted in the mult computation in
timekeeping_adjust() and folded into the ntp_error update in
logarithmic_accumulation(). That keeps the base rate correct and leaves
NTP its full symmetric +/-MAXFREQ range rather than +373/-627 PPM, but
the NTP code in ntp.c never sees it: tick_length is computed without the
correction, so ntp.c's notion of how long a tick is disagrees with the
rate timekeeping actually produces.
Make the offset an explicit part of the NTP tick_length instead. Add
ntp_data::cs_tick_adj, a fixed per-second addend that
ntp_update_frequency() includes alongside ntp_tick_adj and time_freq.
tk_setup_internals() computes it from the difference between the real
cycle_interval duration and the nominal interval, stores it in the
timekeeper, and hands it to NTP through a new argument to ntp_clear() --
which already recomputes the frequency and is invoked after every
clocksource (re)configuration. timekeeping_init() now uses TK_UPDATE_ALL
for this; clearing NTP there is otherwise redundant since ntp_init() has
just initialised it.
ntp.c now computes the true tick rate, giving a single source of truth.
Like ntp_tick_adj, cs_tick_adj stays internal to the kernel: userspace
still sees the nominal 1.000000 ms tick via adjtimex and is unaware of
the addends. timekeeping_adjust() and logarithmic_accumulation() use
ntp_tick / xtime_interval directly, and xtime_remainder is removed.
The base-rate arithmetic is unchanged: ntp_tick becomes
xtime_interval << ntp_error_shift, so the mult division yields the same
base mult and the ntp_error accumulation still nets to zero per tick.
Beyond the cleanup of treating all the tick_length contributions
(nominal interval, ntp_tick_adj, cs_tick_adj, time_freq) consistently
as addends in one place, it also prepares for feed-forward discipline:
a future timekeeping_set_reference() will set tick_length to track an
absolute external reference such as a vmclock, and that path needs
ntp.c to own a tick_length that already reflects the clocksource
quantisation, with no hidden correction applied elsewhere.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.8
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260621220051.1030462-4-dwmw2@infradead.org
Remove "_minmax" from proc_doulongvec_minmax_conv as it does not enforce
min/max limits but serves as a generic converter for unsigned long
vectors. Update function declaration in sysctl.h, definition in
sysctl.c, and caller in jiffies.c accordingly.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
The existing do_proc_doulongvec_minmax conversions (based on conv{mul,div})
are replaced with a call to a converter callback that is passed by the
caller.
Replace the values (HZ, 1000l) passed to proc_doulongvec_minmax_conv in
jiffies.c with a new callback containing millisecond to jiffie
conversion (do_proc_ulong_conv_ms_jiffies). This effectively changes the
simple calculation based on HZ and 1000l to a more robust conversion
based on {_,}_msecs_to_jiffies.
Change specifics
================
* sysctl.h API
- Implement new ulong uni & bi-directional converters (proc_ulong_*);
export them so they can be used in proc_doulongvec_ms_jiffies_minmax
(jiffies.c).
- Replace two arguments (conv{mul,div}) in proc_doulongvec_minmax_conv
with a general converter callback function that will be forwarded to
do_proc_doulongvec.
* do_proc_doulongvec
- Replace the hardcoded uni-directional converters with a call to the
call back converter function
- Generate do_proc_doulongvec with do_proc_dotypevec macro
- Rename do_proc_doulongvec_minmax to do_proc_doulongvec
* jiffies
- Create uni and bi-directional converters for milliseconds to jiffies
(sysctl_{u2k,k2u}_ulong_conv_ms, do_proc_ulong_conv_ms_jiffies)
- Pass the new bi-directional converter to proc_doulongvec_minmax_conv.
Signed-off-by: Joel Granados <joel.granados@kernel.org>
timekeeping_apply_adjustment() modifies xtime_nsec to ensure monotonicity
when mult changes:
xtime_nsec -= offset
This ensures that the time reported to userspace does not jump when the
multiplier is adjusted from one tick to the next. However, the ntp_error
accumulator which tracks the difference between intended and actual
clock position was not being updated to reflect this additional
discrepancy.
An earlier attempt at this compensation existed as:
ntp_error -= (interval - offset) << ntp_error_shift
but was removed in commit c2cda2a5bd ("timekeeping/ntp: Don't align
NTP frequency adjustments to ticks") because it was a major source of
NTP error. That's because (interval - offset) was wrong: the subtraction
of "interval" prematurely accounted for the changed xtime_interval of
the next tick, which would be correctly accounted in the next
accumulation anyway — a double subtraction.
What is actually needed is just the "offset" part: ntp_error must be
told that xtime_nsec moved by "offset" without a corresponding change
in the intended position. For the normal ±1 mult dithering this is
negligible (the adjustments cancel over time), but for larger mult
changes — such as when an external reference clock sets a new
frequency — the one-time uncompensated offset is significant.
Fix by adjusting ntp_error by the correct amount:
ntp_error += offset << ntp_error_shift
This keeps ntp_error consistent with the actual xtime_nsec position
after the adjustment, and ensures the discrepancy is correctly smoothed
away over time and the clock returns to where it should have been.
Fixes: c2cda2a5bd ("timekeeping/ntp: Don't align NTP frequency adjustments to ticks")
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.6-1m
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260621220051.1030462-3-dwmw2@infradead.org
The fault handler for the timens page does not have access to the target
task and therefore can not be invoked remotely.
Currently the handler relies on the fact that the vvar mapping is marked as
VM_IO and VM_PFNMAP for which the mm core always prevents remote access.
However the VM_IO and VM_PFNMAP flags are going to be removed.
Add an explicit check to prevent remote access to the mapping.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Nam Cao <namcao@linutronix.de>
Link: https://patch.msgid.link/20260630-vdso-mlockall-v4-4-6c93708ce723@linutronix.de
