Merge tag 'mm-stable-2025-07-30-15-25' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:
 "As usual, many cleanups. The below blurbiage describes 42 patchsets.
  21 of those are partially or fully cleanup work. "cleans up",
  "cleanup", "maintainability", "rationalizes", etc.

  I never knew the MM code was so dirty.

  "mm: ksm: prevent KSM from breaking merging of new VMAs" (Lorenzo Stoakes)
     addresses an issue with KSM's PR_SET_MEMORY_MERGE mode: newly
     mapped VMAs were not eligible for merging with existing adjacent
     VMAs.

  "mm/damon: introduce DAMON_STAT for simple and practical access monitoring" (SeongJae Park)
     adds a new kernel module which simplifies the setup and usage of
     DAMON in production environments.

  "stop passing a writeback_control to swap/shmem writeout" (Christoph Hellwig)
     is a cleanup to the writeback code which removes a couple of
     pointers from struct writeback_control.

  "drivers/base/node.c: optimization and cleanups" (Donet Tom)
     contains largely uncorrelated cleanups to the NUMA node setup and
     management code.

  "mm: userfaultfd: assorted fixes and cleanups" (Tal Zussman)
     does some maintenance work on the userfaultfd code.

  "Readahead tweaks for larger folios" (Ryan Roberts)
     implements some tuneups for pagecache readahead when it is reading
     into order>0 folios.

  "selftests/mm: Tweaks to the cow test" (Mark Brown)
     provides some cleanups and consistency improvements to the
     selftests code.

  "Optimize mremap() for large folios" (Dev Jain)
     does that. A 37% reduction in execution time was measured in a
     memset+mremap+munmap microbenchmark.

  "Remove zero_user()" (Matthew Wilcox)
     expunges zero_user() in favor of the more modern memzero_page().

  "mm/huge_memory: vmf_insert_folio_*() and vmf_insert_pfn_pud() fixes" (David Hildenbrand)
     addresses some warts which David noticed in the huge page code.
     These were not known to be causing any issues at this time.

  "mm/damon: use alloc_migrate_target() for DAMOS_MIGRATE_{HOT,COLD" (SeongJae Park)
     provides some cleanup and consolidation work in DAMON.

  "use vm_flags_t consistently" (Lorenzo Stoakes)
     uses vm_flags_t in places where we were inappropriately using other
     types.

  "mm/memfd: Reserve hugetlb folios before allocation" (Vivek Kasireddy)
     increases the reliability of large page allocation in the memfd
     code.

  "mm: Remove pXX_devmap page table bit and pfn_t type" (Alistair Popple)
     removes several now-unneeded PFN_* flags.

  "mm/damon: decouple sysfs from core" (SeongJae Park)
     implememnts some cleanup and maintainability work in the DAMON
     sysfs layer.

  "madvise cleanup" (Lorenzo Stoakes)
     does quite a lot of cleanup/maintenance work in the madvise() code.

  "madvise anon_name cleanups" (Vlastimil Babka)
     provides additional cleanups on top or Lorenzo's effort.

  "Implement numa node notifier" (Oscar Salvador)
     creates a standalone notifier for NUMA node memory state changes.
     Previously these were lumped under the more general memory
     on/offline notifier.

  "Make MIGRATE_ISOLATE a standalone bit" (Zi Yan)
     cleans up the pageblock isolation code and fixes a potential issue
     which doesn't seem to cause any problems in practice.

  "selftests/damon: add python and drgn based DAMON sysfs functionality tests" (SeongJae Park)
     adds additional drgn- and python-based DAMON selftests which are
     more comprehensive than the existing selftest suite.

  "Misc rework on hugetlb faulting path" (Oscar Salvador)
     fixes a rather obscure deadlock in the hugetlb fault code and
     follows that fix with a series of cleanups.

  "cma: factor out allocation logic from __cma_declare_contiguous_nid" (Mike Rapoport)
     rationalizes and cleans up the highmem-specific code in the CMA
     allocator.

  "mm/migration: rework movable_ops page migration (part 1)" (David Hildenbrand)
     provides cleanups and future-preparedness to the migration code.

  "mm/damon: add trace events for auto-tuned monitoring intervals and DAMOS quota" (SeongJae Park)
     adds some tracepoints to some DAMON auto-tuning code.

  "mm/damon: fix misc bugs in DAMON modules" (SeongJae Park)
     does that.

  "mm/damon: misc cleanups" (SeongJae Park)
     also does what it claims.

  "mm: folio_pte_batch() improvements" (David Hildenbrand)
     cleans up the large folio PTE batching code.

  "mm/damon/vaddr: Allow interleaving in migrate_{hot,cold} actions" (SeongJae Park)
     facilitates dynamic alteration of DAMON's inter-node allocation
     policy.

  "Remove unmap_and_put_page()" (Vishal Moola)
     provides a couple of page->folio conversions.

  "mm: per-node proactive reclaim" (Davidlohr Bueso)
     implements a per-node control of proactive reclaim - beyond the
     current memcg-based implementation.

  "mm/damon: remove damon_callback" (SeongJae Park)
     replaces the damon_callback interface with a more general and
     powerful damon_call()+damos_walk() interface.

  "mm/mremap: permit mremap() move of multiple VMAs" (Lorenzo Stoakes)
     implements a number of mremap cleanups (of course) in preparation
     for adding new mremap() functionality: newly permit the remapping
     of multiple VMAs when the user is specifying MREMAP_FIXED. It still
     excludes some specialized situations where this cannot be performed
     reliably.

  "drop hugetlb_free_pgd_range()" (Anthony Yznaga)
     switches some sparc hugetlb code over to the generic version and
     removes the thus-unneeded hugetlb_free_pgd_range().

  "mm/damon/sysfs: support periodic and automated stats update" (SeongJae Park)
     augments the present userspace-requested update of DAMON sysfs
     monitoring files. Automatic update is now provided, along with a
     tunable to control the update interval.

  "Some randome fixes and cleanups to swapfile" (Kemeng Shi)
     does what is claims.

  "mm: introduce snapshot_page" (Luiz Capitulino and David Hildenbrand)
     provides (and uses) a means by which debug-style functions can grab
     a copy of a pageframe and inspect it locklessly without tripping
     over the races inherent in operating on the live pageframe
     directly.

  "use per-vma locks for /proc/pid/maps reads" (Suren Baghdasaryan)
     addresses the large contention issues which can be triggered by
     reads from that procfs file. Latencies are reduced by more than
     half in some situations. The series also introduces several new
     selftests for the /proc/pid/maps interface.

  "__folio_split() clean up" (Zi Yan)
     cleans up __folio_split()!

  "Optimize mprotect() for large folios" (Dev Jain)
     provides some quite large (>3x) speedups to mprotect() when dealing
     with large folios.

  "selftests/mm: reuse FORCE_READ to replace "asm volatile("" : "+r" (XXX));" and some cleanup" (wang lian)
     does some cleanup work in the selftests code.

  "tools/testing: expand mremap testing" (Lorenzo Stoakes)
     extends the mremap() selftest in several ways, including adding
     more checking of Lorenzo's recently added "permit mremap() move of
     multiple VMAs" feature.

  "selftests/damon/sysfs.py: test all parameters" (SeongJae Park)
     extends the DAMON sysfs interface selftest so that it tests all
     possible user-requested parameters. Rather than the present minimal
     subset"

* tag 'mm-stable-2025-07-30-15-25' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (370 commits)
  MAINTAINERS: add missing headers to mempory policy & migration section
  MAINTAINERS: add missing file to cgroup section
  MAINTAINERS: add MM MISC section, add missing files to MISC and CORE
  MAINTAINERS: add missing zsmalloc file
  MAINTAINERS: add missing files to page alloc section
  MAINTAINERS: add missing shrinker files
  MAINTAINERS: move memremap.[ch] to hotplug section
  MAINTAINERS: add missing mm_slot.h file THP section
  MAINTAINERS: add missing interval_tree.c to memory mapping section
  MAINTAINERS: add missing percpu-internal.h file to per-cpu section
  mm/page_alloc: remove trace_mm_alloc_contig_migrate_range_info()
  selftests/damon: introduce _common.sh to host shared function
  selftests/damon/sysfs.py: test runtime reduction of DAMON parameters
  selftests/damon/sysfs.py: test non-default parameters runtime commit
  selftests/damon/sysfs.py: generalize DAMON context commit assertion
  selftests/damon/sysfs.py: generalize monitoring attributes commit assertion
  selftests/damon/sysfs.py: generalize DAMOS schemes commit assertion
  selftests/damon/sysfs.py: test DAMOS filters commitment
  selftests/damon/sysfs.py: generalize DAMOS scheme commit assertion
  selftests/damon/sysfs.py: test DAMOS destinations commitment
  ...
This commit is contained in:
Linus Torvalds
2025-07-31 14:57:54 -07:00
329 changed files with 10740 additions and 5808 deletions
@@ -227,3 +227,12 @@ Contact: Jiaqi Yan <jiaqiyan@google.com>
Description:
Of the raw poisoned pages on a NUMA node, how many pages are
recovered by memory error recovery attempt.
What: /sys/devices/system/node/nodeX/reclaim
Date: June 2025
Contact: Linux Memory Management list <linux-mm@kvack.org>
Description:
Perform user-triggered proactive reclaim on a NUMA node.
This interface is equivalent to the memcg variant.
See Documentation/admin-guide/cgroup-v2.rst
@@ -44,6 +44,13 @@ Contact: SeongJae Park <sj@kernel.org>
Description: Reading this file returns the pid of the kdamond if it is
running.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/refresh_ms
Date: Jul 2025
Contact: SeongJae Park <sj@kernel.org>
Description: Writing a value to this file sets the time interval for
automatic DAMON status file contents update. Writing '0'
disables the update. Reading this file returns the value.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/nr_contexts
Date: Mar 2022
Contact: SeongJae Park <sj@kernel.org>
@@ -431,6 +438,28 @@ Description: Directory for DAMON operations set layer-handled DAMOS filters.
/sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/filters
directory.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/dests/nr_dests
Date: Jul 2025
Contact: SeongJae Park <sj@kernel.org>
Description: Writing a number 'N' to this file creates the number of
directories for setting action destinations of the scheme named
'0' to 'N-1' under the dests/ directory.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/dests/<D>/id
Date: Jul 2025
Contact: SeongJae Park <sj@kernel.org>
Description: Writing to and reading from this file sets and gets the id of
the DAMOS action destination. For DAMOS_MIGRATE_{HOT,COLD}
actions, the destination node's node id can be written and
read.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/dests/<D>/weight
Date: Jul 2025
Contact: SeongJae Park <sj@kernel.org>
Description: Writing to and reading from this file sets and gets the weight
of the DAMOS action destination to select as the destination of
each action among the destinations.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/stats/nr_tried
Date: Mar 2022
Contact: SeongJae Park <sj@kernel.org>
@@ -14,3 +14,4 @@ access monitoring and access-aware system operations.
usage
reclaim
lru_sort
stat
@@ -0,0 +1,69 @@
.. SPDX-License-Identifier: GPL-2.0
===================================
Data Access Monitoring Results Stat
===================================
Data Access Monitoring Results Stat (DAMON_STAT) is a static kernel module that
is aimed to be used for simple access pattern monitoring. It monitors accesses
on the system's entire physical memory using DAMON, and provides simplified
access monitoring results statistics, namely idle time percentiles and
estimated memory bandwidth.
Monitoring Accuracy and Overhead
================================
DAMON_STAT uses monitoring intervals :ref:`auto-tuning
<damon_design_monitoring_intervals_autotuning>` to make its accuracy high and
overhead minimum. It auto-tunes the intervals aiming 4 % of observable access
events to be captured in each snapshot, while limiting the resulting sampling
events to be 5 milliseconds in minimum and 10 seconds in maximum. On a few
production server systems, it resulted in consuming only 0.x % single CPU time,
while capturing reasonable quality of access patterns.
Interface: Module Parameters
============================
To use this feature, you should first ensure your system is running on a kernel
that is built with ``CONFIG_DAMON_STAT=y``. The feature can be enabled by
default at build time, by setting ``CONFIG_DAMON_STAT_ENABLED_DEFAULT`` true.
To let sysadmins enable or disable it at boot and/or runtime, and read the
monitoring results, DAMON_STAT provides module parameters. Following
sections are descriptions of the parameters.
enabled
-------
Enable or disable DAMON_STAT.
You can enable DAMON_STAT by setting the value of this parameter as ``Y``.
Setting it as ``N`` disables DAMON_STAT. The default value is set by
``CONFIG_DAMON_STAT_ENABLED_DEFAULT`` build config option.
estimated_memory_bandwidth
--------------------------
Estimated memory bandwidth consumption (bytes per second) of the system.
DAMON_STAT reads observed access events on the current DAMON results snapshot
and converts it to memory bandwidth consumption estimation in bytes per second.
The resulting metric is exposed to user via this read-only parameter. Because
DAMON uses sampling, this is only an estimation of the access intensity rather
than accurate memory bandwidth.
memory_idle_ms_percentiles
--------------------------
Per-byte idle time (milliseconds) percentiles of the system.
DAMON_STAT calculates how long each byte of the memory was not accessed until
now (idle time), based on the current DAMON results snapshot. If DAMON found a
region of access frequency (nr_accesses) larger than zero, every byte of the
region gets zero idle time. If a region has zero access frequency
(nr_accesses), how long the region was keeping the zero access frequency (age)
becomes the idle time of every byte of the region. Then, DAMON_STAT exposes
the percentiles of the idle time values via this read-only parameter. Reading
the parameter returns 101 idle time values in milliseconds, separated by comma.
Each value represents 0-th, 1st, 2nd, 3rd, ..., 99th and 100th percentile idle
times.
+39 -7
View File
@@ -59,7 +59,7 @@ comma (",").
:ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
:ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
│ │ :ref:`0 <sysfs_kdamond>`/state,pid
│ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
│ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
│ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations
│ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
@@ -85,6 +85,8 @@ comma (",").
│ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
│ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
│ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max
│ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
│ │ │ │ │ │ │ │ 0/id,weight
│ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds
│ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
│ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
@@ -121,8 +123,8 @@ kdamond.
kdamonds/<N>/
-------------
In each kdamond directory, two files (``state`` and ``pid``) and one directory
(``contexts``) exist.
In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
and one directory (``contexts``) exist.
Reading ``state`` returns ``on`` if the kdamond is currently running, or
``off`` if it is not running.
@@ -159,6 +161,13 @@ Users can write below commands for the kdamond to the ``state`` file.
If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
Users can ask the kernel to periodically update files showing auto-tuned
parameters and DAMOS stats instead of manually writing
``update_tuned_intervals`` like keywords to ``state`` file. For this, users
should write the desired update time interval in milliseconds to ``refresh_ms``
file. If the interval is zero, the periodic update is disabled. Reading the
file shows currently set time interval.
``contexts`` directory contains files for controlling the monitoring contexts
that this kdamond will execute.
@@ -307,10 +316,10 @@ to ``N-1``. Each directory represents each DAMON-based operation scheme.
schemes/<N>/
------------
In each scheme directory, seven directories (``access_pattern``, ``quotas``,
``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``stats``, and
``tried_regions``) and three files (``action``, ``target_nid`` and
``apply_interval``) exist.
In each scheme directory, eight directories (``access_pattern``, ``quotas``,
``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
and ``apply_interval``) exist.
The ``action`` file is for setting and getting the scheme's :ref:`action
<damon_design_damos_action>`. The keywords that can be written to and read
@@ -484,6 +493,29 @@ Refer to the :ref:`DAMOS filters design documentation
of different ``allow`` works, when each of the filters are supported, and
differences on stats.
.. _damon_sysfs_dests:
schemes/<N>/dests/
------------------
Directory for specifying the destinations of given DAMON-based operation
scheme's action. This directory is ignored if the action of the given scheme
is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}``
actions are supporting multiple destinations.
In the beginning, the directory has only one file, ``nr_dests``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each action destination.
Each destination directory contains two files, namely ``id`` and ``weight``.
Users can write and read the identifier of the destination to ``id`` file.
For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node
id should be written to ``id`` file. Users can write and read the weight of
the destination among the given destinations to the ``weight`` file. The
weight can be an arbitrary integer. When DAMOS apply the action to each entity
of the memory region, it will select the destination of the action based on the
relative weights of the destinations.
.. _sysfs_schemes_stats:
schemes/<N>/stats/
+15 -4
View File
@@ -107,7 +107,7 @@ sysfs
Global THP controls
-------------------
Transparent Hugepage Support for anonymous memory can be entirely disabled
Transparent Hugepage Support for anonymous memory can be disabled
(mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE
regions (to avoid the risk of consuming more memory resources) or enabled
system wide. This can be achieved per-supported-THP-size with one of::
@@ -119,6 +119,11 @@ system wide. This can be achieved per-supported-THP-size with one of::
where <size> is the hugepage size being addressed, the available sizes
for which vary by system.
.. note:: Setting "never" in all sysfs THP controls does **not** disable
Transparent Huge Pages globally. This is because ``madvise(...,
MADV_COLLAPSE)`` ignores these settings and collapses ranges to
PMD-sized huge pages unconditionally.
For example::
echo always >/sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled
@@ -187,7 +192,9 @@ madvise
behaviour.
never
should be self-explanatory.
should be self-explanatory. Note that ``madvise(...,
MADV_COLLAPSE)`` can still cause transparent huge pages to be
obtained even if this mode is specified everywhere.
By default kernel tries to use huge, PMD-mappable zero page on read
page fault to anonymous mapping. It's possible to disable huge zero
@@ -378,7 +385,9 @@ always
Attempt to allocate huge pages every time we need a new page;
never
Do not allocate huge pages;
Do not allocate huge pages. Note that ``madvise(..., MADV_COLLAPSE)``
can still cause transparent huge pages to be obtained even if this mode
is specified everywhere;
within_size
Only allocate huge page if it will be fully within i_size.
@@ -434,7 +443,9 @@ inherit
have enabled="inherit" and all other hugepage sizes have enabled="never";
never
Do not allocate <size> huge pages;
Do not allocate <size> huge pages. Note that ``madvise(...,
MADV_COLLAPSE)`` can still cause transparent huge pages to be obtained
even if this mode is specified everywhere;
within_size
Only allocate <size> huge page if it will be fully within i_size.
+82 -9
View File
@@ -9,6 +9,9 @@ Memory hotplug event notifier
Hotplugging events are sent to a notification queue.
Memory notifier
----------------
There are six types of notification defined in ``include/linux/memory.h``:
MEM_GOING_ONLINE
@@ -56,20 +59,18 @@ The third argument (arg) passes a pointer of struct memory_notify::
struct memory_notify {
unsigned long start_pfn;
unsigned long nr_pages;
int status_change_nid_normal;
int status_change_nid;
}
- start_pfn is start_pfn of online/offline memory.
- nr_pages is # of pages of online/offline memory.
- status_change_nid_normal is set node id when N_NORMAL_MEMORY of nodemask
is (will be) set/clear, if this is -1, then nodemask status is not changed.
- status_change_nid is set node id when N_MEMORY of nodemask is (will be)
set/clear. It means a new(memoryless) node gets new memory by online and a
node loses all memory. If this is -1, then nodemask status is not changed.
If status_changed_nid* >= 0, callback should create/discard structures for the
node if necessary.
It is possible to get notified for MEM_CANCEL_ONLINE without having been notified
for MEM_GOING_ONLINE, and the same applies to MEM_CANCEL_OFFLINE and
MEM_GOING_OFFLINE.
This can happen when a consumer fails, meaning we break the callchain and we
stop calling the remaining consumers of the notifier.
It is then important that users of memory_notify make no assumptions and get
prepared to handle such cases.
The callback routine shall return one of the values
NOTIFY_DONE, NOTIFY_OK, NOTIFY_BAD, NOTIFY_STOP
@@ -83,6 +84,78 @@ further processing of the notification queue.
NOTIFY_STOP stops further processing of the notification queue.
Numa node notifier
------------------
There are six types of notification defined in ``include/linux/node.h``:
NODE_ADDING_FIRST_MEMORY
Generated before memory becomes available to this node for the first time.
NODE_CANCEL_ADDING_FIRST_MEMORY
Generated if NODE_ADDING_FIRST_MEMORY fails.
NODE_ADDED_FIRST_MEMORY
Generated when memory has become available fo this node for the first time.
NODE_REMOVING_LAST_MEMORY
Generated when the last memory available to this node is about to be offlined.
NODE_CANCEL_REMOVING_LAST_MEMORY
Generated when NODE_CANCEL_REMOVING_LAST_MEMORY fails.
NODE_REMOVED_LAST_MEMORY
Generated when the last memory available to this node has been offlined.
A callback routine can be registered by calling::
hotplug_node_notifier(callback_func, priority)
Callback functions with higher values of priority are called before callback
functions with lower values.
A callback function must have the following prototype::
int callback_func(
struct notifier_block *self, unsigned long action, void *arg);
The first argument of the callback function (self) is a pointer to the block
of the notifier chain that points to the callback function itself.
The second argument (action) is one of the event types described above.
The third argument (arg) passes a pointer of struct node_notify::
struct node_notify {
int nid;
}
- nid is the node we are adding or removing memory to.
It is possible to get notified for NODE_CANCEL_ADDING_FIRST_MEMORY without
having been notified for NODE_ADDING_FIRST_MEMORY, and the same applies to
NODE_CANCEL_REMOVING_LAST_MEMORY and NODE_REMOVING_LAST_MEMORY.
This can happen when a consumer fails, meaning we break the callchain and we
stop calling the remaining consumers of the notifier.
It is then important that users of node_notify make no assumptions and get
prepared to handle such cases.
The callback routine shall return one of the values
NOTIFY_DONE, NOTIFY_OK, NOTIFY_BAD, NOTIFY_STOP
defined in ``include/linux/notifier.h``
NOTIFY_DONE and NOTIFY_OK have no effect on the further processing.
NOTIFY_BAD is used as response to the NODE_ADDING_FIRST_MEMORY,
NODE_REMOVING_LAST_MEMORY, NODE_ADDED_FIRST_MEMORY or
NODE_REMOVED_LAST_MEMORY action to cancel hotplugging.
It stops further processing of the notification queue.
NOTIFY_STOP stops further processing of the notification queue.
Please note that we should not fail for NODE_ADDED_FIRST_MEMORY /
NODE_REMOVED_FIRST_MEMORY, as memory_hotplug code cannot rollback at that
point anymore.
Locking Internals
=================
+5 -3
View File
@@ -1196,12 +1196,14 @@ SecPageTables
Memory consumed by secondary page tables, this currently includes
KVM mmu and IOMMU allocations on x86 and arm64.
NFS_Unstable
Always zero. Previous counted pages which had been written to
Always zero. Previously counted pages which had been written to
the server, but has not been committed to stable storage.
Bounce
Memory used for block device "bounce buffers"
Always zero. Previously memory used for block device
"bounce buffers".
WritebackTmp
Memory used by FUSE for temporary writeback buffers
Always zero. Previously memory used by FUSE for temporary
writeback buffers.
CommitLimit
Based on the overcommit ratio ('vm.overcommit_ratio'),
this is the total amount of memory currently available to
+4 -10
View File
@@ -30,8 +30,6 @@ PTE Page Table Helpers
+---------------------------+--------------------------------------------------+
| pte_protnone | Tests a PROT_NONE PTE |
+---------------------------+--------------------------------------------------+
| pte_devmap | Tests a ZONE_DEVICE mapped PTE |
+---------------------------+--------------------------------------------------+
| pte_soft_dirty | Tests a soft dirty PTE |
+---------------------------+--------------------------------------------------+
| pte_swp_soft_dirty | Tests a soft dirty swapped PTE |
@@ -104,8 +102,6 @@ PMD Page Table Helpers
+---------------------------+--------------------------------------------------+
| pmd_protnone | Tests a PROT_NONE PMD |
+---------------------------+--------------------------------------------------+
| pmd_devmap | Tests a ZONE_DEVICE mapped PMD |
+---------------------------+--------------------------------------------------+
| pmd_soft_dirty | Tests a soft dirty PMD |
+---------------------------+--------------------------------------------------+
| pmd_swp_soft_dirty | Tests a soft dirty swapped PMD |
@@ -177,8 +173,6 @@ PUD Page Table Helpers
+---------------------------+--------------------------------------------------+
| pud_write | Tests a writable PUD |
+---------------------------+--------------------------------------------------+
| pud_devmap | Tests a ZONE_DEVICE mapped PUD |
+---------------------------+--------------------------------------------------+
| pud_mkyoung | Creates a young PUD |
+---------------------------+--------------------------------------------------+
| pud_mkold | Creates an old PUD |
@@ -242,13 +236,13 @@ SWAP Page Table Helpers
========================
+---------------------------+--------------------------------------------------+
| __pte_to_swp_entry | Creates a swapped entry (arch) from a mapped PTE |
| __pte_to_swp_entry | Creates a swp_entry_t (arch) from a swap PTE |
+---------------------------+--------------------------------------------------+
| __swp_to_pte_entry | Creates a mapped PTE from a swapped entry (arch) |
| __swp_entry_to_pte | Creates a swap PTE from a swp_entry_t (arch) |
+---------------------------+--------------------------------------------------+
| __pmd_to_swp_entry | Creates a swapped entry (arch) from a mapped PMD |
| __pmd_to_swp_entry | Creates a swp_entry_t (arch) from a swap PMD |
+---------------------------+--------------------------------------------------+
| __swp_to_pmd_entry | Creates a mapped PMD from a swapped entry (arch) |
| __swp_entry_to_pmd | Creates a swap PMD from a swp_entry_t (arch) |
+---------------------------+--------------------------------------------------+
| is_migration_entry | Tests a migration (read or write) swapped entry |
+-------------------------------+----------------------------------------------+
+2 -2
View File
@@ -452,9 +452,9 @@ that supports each action are as below.
- ``lru_deprio``: Deprioritize the region on its LRU lists.
Supported by ``paddr`` operations set.
- ``migrate_hot``: Migrate the regions prioritizing warmer regions.
Supported by ``paddr`` operations set.
Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set.
- ``migrate_cold``: Migrate the regions prioritizing colder regions.
Supported by ``paddr`` operations set.
Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set.
- ``stat``: Do nothing but count the statistics.
Supported by all operations sets.
+18 -15
View File
@@ -7,9 +7,9 @@ The DAMON subsystem covers the files that are listed in 'DATA ACCESS MONITOR'
section of 'MAINTAINERS' file.
The mailing lists for the subsystem are damon@lists.linux.dev and
linux-mm@kvack.org. Patches should be made against the `mm-unstable tree
<https://git.kernel.org/akpm/mm/h/mm-unstable>`_ whenever possible and posted
to the mailing lists.
linux-mm@kvack.org. Patches should be made against the `mm-new tree
<https://git.kernel.org/akpm/mm/h/mm-new>`_ whenever possible and posted to the
mailing lists.
SCM Trees
---------
@@ -17,17 +17,19 @@ SCM Trees
There are multiple Linux trees for DAMON development. Patches under
development or testing are queued in `damon/next
<https://git.kernel.org/sj/h/damon/next>`_ by the DAMON maintainer.
Sufficiently reviewed patches will be queued in `mm-unstable
<https://git.kernel.org/akpm/mm/h/mm-unstable>`_ by the memory management
subsystem maintainer. After more sufficient tests, the patches will be queued
in `mm-stable <https://git.kernel.org/akpm/mm/h/mm-stable>`_, and finally
pull-requested to the mainline by the memory management subsystem maintainer.
Sufficiently reviewed patches will be queued in `mm-new
<https://git.kernel.org/akpm/mm/h/mm-new>`_ by the memory management subsystem
maintainer. As more sufficient tests are done, the patches will move to
`mm-unstable <https://git.kernel.org/akpm/mm/h/mm-unstable>`_ and then to
`mm-stable <https://git.kernel.org/akpm/mm/h/mm-stable>`_. And finally those
will be pull-requested to the mainline by the memory management subsystem
maintainer.
Note again the patches for `mm-unstable tree
<https://git.kernel.org/akpm/mm/h/mm-unstable>`_ are queued by the memory
management subsystem maintainer. If the patches requires some patches in
`damon/next tree <https://git.kernel.org/sj/h/damon/next>`_ which not yet merged
in mm-unstable, please make sure the requirement is clearly specified.
Note again the patches for `mm-new tree
<https://git.kernel.org/akpm/mm/h/mm-new>`_ are queued by the memory management
subsystem maintainer. If the patches requires some patches in `damon/next tree
<https://git.kernel.org/sj/h/damon/next>`_ which not yet merged in mm-new,
please make sure the requirement is clearly specified.
Submit checklist addendum
-------------------------
@@ -53,8 +55,9 @@ Further doing below and putting the results will be helpful.
Key cycle dates
---------------
Patches can be sent anytime. Key cycle dates of the `mm-unstable
<https://git.kernel.org/akpm/mm/h/mm-unstable>`_ and `mm-stable
Patches can be sent anytime. Key cycle dates of the `mm-new
<https://git.kernel.org/akpm/mm/h/mm-new>`_, `mm-unstable
<https://git.kernel.org/akpm/mm/h/mm-unstable>`_and `mm-stable
<https://git.kernel.org/akpm/mm/h/mm-stable>`_ trees depend on the memory
management subsystem maintainer.
+25 -10
View File
@@ -146,18 +146,33 @@ Steps:
18. The new page is moved to the LRU and can be scanned by the swapper,
etc. again.
Non-LRU page migration
======================
movable_ops page migration
==========================
Although migration originally aimed for reducing the latency of memory
accesses for NUMA, compaction also uses migration to create high-order
pages. For compaction purposes, it is also useful to be able to move
non-LRU pages, such as zsmalloc and virtio-balloon pages.
Selected typed, non-folio pages (e.g., pages inflated in a memory balloon,
zsmalloc pages) can be migrated using the movable_ops migration framework.
If a driver wants to make its pages movable, it should define a struct
movable_operations. It then needs to call __SetPageMovable() on each
page that it may be able to move. This uses the ``page->mapping`` field,
so this field is not available for the driver to use for other purposes.
The "struct movable_operations" provide callbacks specific to a page type
for isolating, migrating and un-isolating (putback) these pages.
Once a page is indicated as having movable_ops, that condition must not
change until the page was freed back to the buddy. This includes not
changing/clearing the page type and not changing/clearing the
PG_movable_ops page flag.
Arbitrary drivers cannot currently make use of this framework, as it
requires:
(a) a page type
(b) indicating them as possibly having movable_ops in page_has_movable_ops()
based on the page type
(c) returning the movable_ops from page_movable_ops() based on the page
type
(d) not reusing the PG_movable_ops and PG_movable_ops_isolated page flags
for other purposes
For example, balloon drivers can make use of this framework through the
balloon-compaction infrastructure residing in the core kernel.
Monitoring Migration
=====================
+1 -1
View File
@@ -584,7 +584,7 @@ Compaction control
``compact_blockskip_flush``
Set to true when compaction migration scanner and free scanner meet, which
means the ``PB_migrate_skip`` bits should be cleared.
means the ``PB_compact_skip`` bits should be cleared.
``contiguous``
Set to true when the zone is contiguous (in other words, no hole).
+48 -6
View File
@@ -303,7 +303,9 @@ There are four key operations typically performed on page tables:
1. **Traversing** page tables - Simply reading page tables in order to traverse
them. This only requires that the VMA is kept stable, so a lock which
establishes this suffices for traversal (there are also lockless variants
which eliminate even this requirement, such as :c:func:`!gup_fast`).
which eliminate even this requirement, such as :c:func:`!gup_fast`). There is
also a special case of page table traversal for non-VMA regions which we
consider separately below.
2. **Installing** page table mappings - Whether creating a new mapping or
modifying an existing one in such a way as to change its identity. This
requires that the VMA is kept stable via an mmap or VMA lock (explicitly not
@@ -335,15 +337,13 @@ ahead and perform these operations on page tables (though internally, kernel
operations that perform writes also acquire internal page table locks to
serialise - see the page table implementation detail section for more details).
.. note:: We free empty PTE tables on zap under the RCU lock - this does not
change the aforementioned locking requirements around zapping.
When **installing** page table entries, the mmap or VMA lock must be held to
keep the VMA stable. We explore why this is in the page table locking details
section below.
.. warning:: Page tables are normally only traversed in regions covered by VMAs.
If you want to traverse page tables in areas that might not be
covered by VMAs, heavier locking is required.
See :c:func:`!walk_page_range_novma` for details.
**Freeing** page tables is an entirely internal memory management operation and
has special requirements (see the page freeing section below for more details).
@@ -355,6 +355,44 @@ has special requirements (see the page freeing section below for more details).
from the reverse mappings, but no other VMAs can be permitted to be
accessible and span the specified range.
Traversing non-VMA page tables
------------------------------
We've focused above on traversal of page tables belonging to VMAs. It is also
possible to traverse page tables which are not represented by VMAs.
Kernel page table mappings themselves are generally managed but whatever part of
the kernel established them and the aforementioned locking rules do not apply -
for instance vmalloc has its own set of locks which are utilised for
establishing and tearing down page its page tables.
However, for convenience we provide the :c:func:`!walk_kernel_page_table_range`
function which is synchronised via the mmap lock on the :c:macro:`!init_mm`
kernel instantiation of the :c:struct:`!struct mm_struct` metadata object.
If an operation requires exclusive access, a write lock is used, but if not, a
read lock suffices - we assert only that at least a read lock has been acquired.
Since, aside from vmalloc and memory hot plug, kernel page tables are not torn
down all that often - this usually suffices, however any caller of this
functionality must ensure that any additionally required locks are acquired in
advance.
We also permit a truly unusual case is the traversal of non-VMA ranges in
**userland** ranges, as provided for by :c:func:`!walk_page_range_debug`.
This has only one user - the general page table dumping logic (implemented in
:c:macro:`!mm/ptdump.c`) - which seeks to expose all mappings for debug purposes
even if they are highly unusual (possibly architecture-specific) and are not
backed by a VMA.
We must take great care in this case, as the :c:func:`!munmap` implementation
detaches VMAs under an mmap write lock before tearing down page tables under a
downgraded mmap read lock.
This means such an operation could race with this, and thus an mmap **write**
lock is required.
Lock ordering
-------------
@@ -461,6 +499,10 @@ Locking Implementation Details
Page table locking details
--------------------------
.. note:: This section explores page table locking requirements for page tables
encompassed by a VMA. See the above section on non-VMA page table
traversal for details on how we handle that case.
In addition to the locks described in the terminology section above, we have
additional locks dedicated to page tables:
@@ -62,7 +62,6 @@ memory_notify结构体的指针::
struct memory_notify {
unsigned long start_pfn;
unsigned long nr_pages;
int status_change_nid_normal;
int status_change_nid;
}
@@ -70,8 +69,6 @@ memory_notify结构体的指针::
- nr_pages是在线/离线内存的页数。
- status_change_nid_normal是当nodemask的N_NORMAL_MEMORY被设置/清除时设置节
点id,如果是-1,则nodemask状态不改变。
- status_change_nid是当nodemask的N_MEMORY被(将)设置/清除时设置的节点id。这
意味着一个新的(没上线的)节点通过联机获得新的内存,而一个节点失去了所有的内
+79 -15
View File
@@ -6269,9 +6269,11 @@ L: cgroups@vger.kernel.org
L: linux-mm@kvack.org
S: Maintained
F: include/linux/memcontrol.h
F: include/linux/page_counter.h
F: mm/memcontrol.c
F: mm/memcontrol-v1.c
F: mm/memcontrol-v1.h
F: mm/page_counter.c
F: mm/swap_cgroup.c
F: samples/cgroup/*
F: tools/testing/selftests/cgroup/memcg_protection.m
@@ -15928,6 +15930,8 @@ F: Documentation/admin-guide/mm/memory-hotplug.rst
F: Documentation/core-api/memory-hotplug.rst
F: drivers/base/memory.c
F: include/linux/memory_hotplug.h
F: include/linux/memremap.h
F: mm/memremap.c
F: mm/memory_hotplug.c
F: tools/testing/selftests/memory-hotplug/
@@ -15938,23 +15942,8 @@ S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
T: quilt git://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new
F: Documentation/admin-guide/mm/
F: Documentation/mm/
F: include/linux/gfp.h
F: include/linux/gfp_types.h
F: include/linux/memory_hotplug.h
F: include/linux/memory-tiers.h
F: include/linux/mempolicy.h
F: include/linux/mempool.h
F: include/linux/memremap.h
F: include/linux/mmzone.h
F: include/linux/mmu_notifier.h
F: include/linux/pagewalk.h
F: include/trace/events/ksm.h
F: mm/
F: tools/mm/
F: tools/testing/selftests/mm/
N: include/linux/page[-_]*
MEMORY MANAGEMENT - CORE
M: Andrew Morton <akpm@linux-foundation.org>
@@ -15969,18 +15958,40 @@ L: linux-mm@kvack.org
S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: include/linux/gfp.h
F: include/linux/gfp_types.h
F: include/linux/highmem.h
F: include/linux/memory.h
F: include/linux/mm.h
F: include/linux/mm_*.h
F: include/linux/mmzone.h
F: include/linux/mmdebug.h
F: include/linux/mmu_notifier.h
F: include/linux/pagewalk.h
F: include/linux/pgtable.h
F: include/linux/ptdump.h
F: include/linux/vmpressure.h
F: include/linux/vmstat.h
F: kernel/fork.c
F: mm/Kconfig
F: mm/debug.c
F: mm/folio-compat.c
F: mm/highmem.c
F: mm/init-mm.c
F: mm/internal.h
F: mm/maccess.c
F: mm/memory.c
F: mm/mmu_notifier.c
F: mm/mmzone.c
F: mm/pagewalk.c
F: mm/pgtable-generic.c
F: mm/ptdump.c
F: mm/sparse-vmemmap.c
F: mm/sparse.c
F: mm/util.c
F: mm/vmpressure.c
F: mm/vmstat.c
N: include/linux/page[-_]*
MEMORY MANAGEMENT - EXECMEM
M: Andrew Morton <akpm@linux-foundation.org>
@@ -16020,6 +16031,7 @@ F: Documentation/mm/ksm.rst
F: include/linux/ksm.h
F: include/trace/events/ksm.h
F: mm/ksm.c
F: mm/mm_slot.h
MEMORY MANAGEMENT - MEMORY POLICY AND MIGRATION
M: Andrew Morton <akpm@linux-foundation.org>
@@ -16037,11 +16049,49 @@ S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: include/linux/mempolicy.h
F: include/uapi/linux/mempolicy.h
F: include/linux/migrate.h
F: include/linux/migrate_mode.h
F: mm/mempolicy.c
F: mm/migrate.c
F: mm/migrate_device.c
MEMORY MANAGEMENT - MISC
M: Andrew Morton <akpm@linux-foundation.org>
M: David Hildenbrand <david@redhat.com>
R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
R: Liam R. Howlett <Liam.Howlett@oracle.com>
R: Vlastimil Babka <vbabka@suse.cz>
R: Mike Rapoport <rppt@kernel.org>
R: Suren Baghdasaryan <surenb@google.com>
R: Michal Hocko <mhocko@suse.com>
L: linux-mm@kvack.org
S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: Documentation/admin-guide/mm/
F: Documentation/mm/
F: include/linux/cma.h
F: include/linux/dmapool.h
F: include/linux/ioremap.h
F: include/linux/memory-tiers.h
F: include/linux/page_idle.h
F: mm/backing-dev.c
F: mm/cma.c
F: mm/cma_debug.c
F: mm/cma_sysfs.c
F: mm/dmapool.c
F: mm/dmapool_test.c
F: mm/early_ioremap.c
F: mm/fadvise.c
F: mm/ioremap.c
F: mm/mapping_dirty_helpers.c
F: mm/memory-tiers.c
F: mm/page_idle.c
F: mm/pgalloc-track.h
F: mm/process_vm_access.c
F: tools/testing/selftests/mm/
MEMORY MANAGEMENT - NUMA MEMBLOCKS AND NUMA EMULATION
M: Andrew Morton <akpm@linux-foundation.org>
M: Mike Rapoport <rppt@kernel.org>
@@ -16078,6 +16128,7 @@ F: include/linux/gfp.h
F: include/linux/page-isolation.h
F: mm/compaction.c
F: mm/debug_page_alloc.c
F: mm/debug_page_ref.c
F: mm/fail_page_alloc.c
F: mm/page_alloc.c
F: mm/page_ext.c
@@ -16086,8 +16137,10 @@ F: mm/page_isolation.c
F: mm/page_owner.c
F: mm/page_poison.c
F: mm/page_reporting.c
F: mm/page_reporting.h
F: mm/show_mem.c
F: mm/shuffle.c
F: mm/shuffle.h
MEMORY MANAGEMENT - RECLAIM
M: Andrew Morton <akpm@linux-foundation.org>
@@ -16165,6 +16218,7 @@ F: include/linux/khugepaged.h
F: include/trace/events/huge_memory.h
F: mm/huge_memory.c
F: mm/khugepaged.c
F: mm/mm_slot.h
F: tools/testing/selftests/mm/khugepaged.c
F: tools/testing/selftests/mm/split_huge_page_test.c
F: tools/testing/selftests/mm/transhuge-stress.c
@@ -16207,6 +16261,7 @@ S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: include/trace/events/mmap.h
F: mm/interval_tree.c
F: mm/mincore.c
F: mm/mlock.c
F: mm/mmap.c
@@ -19029,6 +19084,11 @@ F: Documentation/mm/page_table_check.rst
F: include/linux/page_table_check.h
F: mm/page_table_check.c
PAGE STATE DEBUG SCRIPT
M: Ye Liu <liuye@kylinos.cn>
S: Maintained
F: tools/mm/show_page_info.py
PANASONIC LAPTOP ACPI EXTRAS DRIVER
M: Kenneth Chan <kenneth.t.chan@gmail.com>
L: platform-driver-x86@vger.kernel.org
@@ -19665,6 +19725,7 @@ F: arch/*/include/asm/percpu.h
F: include/linux/percpu*.h
F: lib/percpu*.c
F: mm/percpu*.c
F: mm/percpu-internal.h
PER-TASK DELAY ACCOUNTING
M: Balbir Singh <bsingharora@gmail.com>
@@ -22866,7 +22927,9 @@ R: Muchun Song <muchun.song@linux.dev>
L: linux-mm@kvack.org
S: Maintained
F: Documentation/admin-guide/mm/shrinker_debugfs.rst
F: include/linux/list_lru.h
F: include/linux/shrinker.h
F: mm/list_lru.c
F: mm/shrinker.c
F: mm/shrinker_debug.c
@@ -27700,6 +27763,7 @@ L: linux-mm@kvack.org
S: Maintained
F: Documentation/mm/zsmalloc.rst
F: include/linux/zsmalloc.h
F: mm/zpdesc.h
F: mm/zsmalloc.c
ZSTD
+1
View File
@@ -7,6 +7,7 @@ config ALPHA
select ARCH_HAS_DMA_OPS if PCI
select ARCH_MIGHT_HAVE_PC_PARPORT
select ARCH_MIGHT_HAVE_PC_SERIO
select ARCH_MODULE_NEEDS_WEAK_PER_CPU if SMP
select ARCH_NO_PREEMPT
select ARCH_NO_SG_CHAIN
select ARCH_USE_CMPXCHG_LOCKREF
+2 -3
View File
@@ -9,10 +9,9 @@
* way above 4G.
*
* Always use weak definitions for percpu variables in modules.
* Therefore, we have enabled CONFIG_ARCH_MODULE_NEEDS_WEAK_PER_CPU
* in the Kconfig.
*/
#if defined(MODULE) && defined(CONFIG_SMP)
#define ARCH_NEEDS_WEAK_PER_CPU
#endif
#include <asm-generic/percpu.h>
+1 -1
View File
@@ -268,7 +268,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
int sig, code;
vm_fault_t fault;
unsigned int flags = FAULT_FLAG_DEFAULT;
unsigned long vm_flags = VM_ACCESS_FLAGS;
vm_flags_t vm_flags = VM_ACCESS_FLAGS;
if (kprobe_page_fault(regs, fsr))
return 0;
-1
View File
@@ -42,7 +42,6 @@ config ARM64
select ARCH_HAS_NONLEAF_PMD_YOUNG if ARM64_HAFT
select ARCH_HAS_PREEMPT_LAZY
select ARCH_HAS_PTDUMP
select ARCH_HAS_PTE_DEVMAP
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_HW_PTE_YOUNG
select ARCH_HAS_SETUP_DMA_OPS

Some files were not shown because too many files have changed in this diff Show More