armbian/linux-rockchip
0
0
Fork 0
mirror of https://github.com/armbian/linux-rockchip.git synced 2026-01-06 11:08:10 -08:00
Code Issues Packages Projects Releases Wiki Activity

Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Browse Source 
Pull arm64 updates from Will Deacon:

 - virt_to_page/page_address optimisations

 - support for NUMA systems described using device-tree

 - support for hibernate/suspend-to-disk

 - proper support for maxcpus= command line parameter

 - detection and graceful handling of AArch64-only CPUs

 - miscellaneous cleanups and non-critical fixes

* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (92 commits)
  arm64: do not enforce strict 16 byte alignment to stack pointer
  arm64: kernel: Fix incorrect brk randomization
  arm64: cpuinfo: Missing NULL terminator in compat_hwcap_str
  arm64: secondary_start_kernel: Remove unnecessary barrier
  arm64: Ensure pmd_present() returns false after pmd_mknotpresent()
  arm64: Replace hard-coded values in the pmd/pud_bad() macros
  arm64: Implement pmdp_set_access_flags() for hardware AF/DBM
  arm64: Fix typo in the pmdp_huge_get_and_clear() definition
  arm64: mm: remove unnecessary EXPORT_SYMBOL_GPL
  arm64: always use STRICT_MM_TYPECHECKS
  arm64: kvm: Fix kvm teardown for systems using the extended idmap
  arm64: kaslr: increase randomization granularity
  arm64: kconfig: drop CONFIG_RTC_LIB dependency
  arm64: make ARCH_SUPPORTS_DEBUG_PAGEALLOC depend on !HIBERNATION
  arm64: hibernate: Refuse to hibernate if the boot cpu is offline
  arm64: kernel: Add support for hibernate/suspend-to-disk
  PM / Hibernate: Call flush_icache_range() on pages restored in-place
  arm64: Add new asm macro copy_page
  arm64: Promote KERNEL_START/KERNEL_END definitions to a header file
  arm64: kernel: Include _AC definition in page.h
  ...
This commit is contained in:
Linus Torvalds
2016-05-16 17:17:24 -07:00
parent fb6363e9f4 e6d9a52543
commit be092017b6
89 changed files with 3306 additions and 1204 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Documentation/arm64/booting.txt
View File

@@ -132,6 +132,10 @@ NOTE: versions prior to v4.6 cannot make use of memory below the
physical offset of the Image so it is recommended that the Image be
placed as close as possible to the start of system RAM.
If an initrd/initramfs is passed to the kernel at boot, it must reside
entirely within a 1 GB aligned physical memory window of up to 32 GB in
size that fully covers the kernel Image as well.
Any memory described to the kernel (even that below the start of the
image) which is not marked as reserved from the kernel (e.g., with a
memreserve region in the device tree) will be considered as available to

275
Documentation/devicetree/bindings/numa.txt Normal file
View File

@@ -0,0 +1,275 @@
==============================================================================
NUMA binding description.
==============================================================================
==============================================================================
1 - Introduction
==============================================================================
Systems employing a Non Uniform Memory Access (NUMA) architecture contain
collections of hardware resources including processors, memory, and I/O buses,
that comprise what is commonly known as a NUMA node.
Processor accesses to memory within the local NUMA node is generally faster
than processor accesses to memory outside of the local NUMA node.
DT defines interfaces that allow the platform to convey NUMA node
topology information to OS.
==============================================================================
2 - numa-node-id
==============================================================================
For the purpose of identification, each NUMA node is associated with a unique
token known as a node id. For the purpose of this binding
a node id is a 32-bit integer.
A device node is associated with a NUMA node by the presence of a
numa-node-id property which contains the node id of the device.
Example:
/* numa node 0 */
numa-node-id = <0>;
/* numa node 1 */
numa-node-id = <1>;
==============================================================================
3 - distance-map
==============================================================================
The optional device tree node distance-map describes the relative
distance (memory latency) between all numa nodes.
- compatible : Should at least contain "numa-distance-map-v1".
- distance-matrix
This property defines a matrix to describe the relative distances
between all numa nodes.
It is represented as a list of node pairs and their relative distance.
Note:
1. Each entry represents distance from first node to second node.
The distances are equal in either direction.
2. The distance from a node to self (local distance) is represented
with value 10 and all internode distance should be represented with
a value greater than 10.
3. distance-matrix should have entries in lexicographical ascending
order of nodes.
4. There must be only one device node distance-map which must
reside in the root node.
5. If the distance-map node is not present, a default
distance-matrix is used.
Example:
4 nodes connected in mesh/ring topology as below,
0_______20______1
| |
| |
20 20
| |
| |
|_______________|
3 20 2
if relative distance for each hop is 20,
then internode distance would be,
0 -> 1 = 20
1 -> 2 = 20
2 -> 3 = 20
3 -> 0 = 20
0 -> 2 = 40
1 -> 3 = 40
and dt presentation for this distance matrix is,
distance-map {
compatible = "numa-distance-map-v1";
distance-matrix = <0 0 10>,
<0 1 20>,
<0 2 40>,
<0 3 20>,
<1 0 20>,
<1 1 10>,
<1 2 20>,
<1 3 40>,
<2 0 40>,
<2 1 20>,
<2 2 10>,
<2 3 20>,
<3 0 20>,
<3 1 40>,
<3 2 20>,
<3 3 10>;
};
==============================================================================
4 - Example dts
==============================================================================
Dual socket system consists of 2 boards connected through ccn bus and
each board having one socket/soc of 8 cpus, memory and pci bus.
memory@c00000 {
device_type = "memory";
reg = <0x0 0xc00000 0x0 0x80000000>;
/* node 0 */
numa-node-id = <0>;
};
memory@10000000000 {
device_type = "memory";
reg = <0x100 0x0 0x0 0x80000000>;
/* node 1 */
numa-node-id = <1>;
};
cpus {
#address-cells = <2>;
#size-cells = <0>;
cpu@0 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x0>;
enable-method = "psci";
/* node 0 */
numa-node-id = <0>;
};
cpu@1 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x1>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@2 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x2>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@3 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x3>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@4 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x4>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@5 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x5>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@6 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x6>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@7 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x7>;
enable-method = "psci";
numa-node-id = <0>;
};
cpu@8 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x8>;
enable-method = "psci";
/* node 1 */
numa-node-id = <1>;
};
cpu@9 {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0x9>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@a {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xa>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@b {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xb>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@c {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xc>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@d {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xd>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@e {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xe>;
enable-method = "psci";
numa-node-id = <1>;
};
cpu@f {
device_type = "cpu";
compatible = "arm,armv8";
reg = <0x0 0xf>;
enable-method = "psci";
numa-node-id = <1>;
};
};
pcie0: pcie0@848000000000 {
compatible = "arm,armv8";
device_type = "pci";
bus-range = <0 255>;
#size-cells = <2>;
#address-cells = <3>;
reg = <0x8480 0x00000000 0 0x10000000>; /* Configuration space */
ranges = <0x03000000 0x8010 0x00000000 0x8010 0x00000000 0x70 0x00000000>;
/* node 0 */
numa-node-id = <0>;
};
pcie1: pcie1@948000000000 {
compatible = "arm,armv8";
device_type = "pci";
bus-range = <0 255>;
#size-cells = <2>;
#address-cells = <3>;
reg = <0x9480 0x00000000 0 0x10000000>; /* Configuration space */
ranges = <0x03000000 0x9010 0x00000000 0x9010 0x00000000 0x70 0x00000000>;
/* node 1 */
numa-node-id = <1>;
};
distance-map {
compatible = "numa-distance-map-v1";
distance-matrix = <0 0 10>,
<0 1 20>,
<1 1 10>;
};

6
Documentation/kernel-parameters.txt
View File

@@ -168,16 +168,18 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
acpi= [HW,ACPI,X86,ARM64]
Advanced Configuration and Power Interface
Format: { force | off | strict | noirq | rsdt |
Format: { force | on | off | strict | noirq | rsdt |
copy_dsdt }
force -- enable ACPI if default was off
on -- enable ACPI but allow fallback to DT [arm64]
off -- disable ACPI if default was on
noirq -- do not use ACPI for IRQ routing
strict -- Be less tolerant of platforms that are not
strictly ACPI specification compliant.
rsdt -- prefer RSDT over (default) XSDT
copy_dsdt -- copy DSDT to memory
For ARM64, ONLY "acpi=off" or "acpi=force" are available
For ARM64, ONLY "acpi=off", "acpi=on" or "acpi=force"
are available
See also Documentation/power/runtime_pm.txt, pci=noacpi

10
arch/arm/include/asm/kvm_host.h
View File

@@ -265,6 +265,15 @@ static inline void __cpu_init_stage2(void)
kvm_call_hyp(__init_stage2_translation);
}
static inline void __cpu_reset_hyp_mode(phys_addr_t boot_pgd_ptr,
phys_addr_t phys_idmap_start)
{
/*
* TODO
* kvm_call_reset(boot_pgd_ptr, phys_idmap_start);
*/
}
static inline int kvm_arch_dev_ioctl_check_extension(long ext)
{
return 0;
@@ -277,7 +286,6 @@ void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot);
struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr);
static inline void kvm_arch_hardware_disable(void) {}
static inline void kvm_arch_hardware_unsetup(void) {}
static inline void kvm_arch_sync_events(struct kvm *kvm) {}
static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}

1
arch/arm/include/asm/kvm_mmu.h
View File

@@ -66,6 +66,7 @@ void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
phys_addr_t kvm_mmu_get_httbr(void);
phys_addr_t kvm_mmu_get_boot_httbr(void);
phys_addr_t kvm_get_idmap_vector(void);
phys_addr_t kvm_get_idmap_start(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);

124
arch/arm/kvm/arm.c
View File

@@ -16,7 +16,6 @@
* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
#include <linux/errno.h>
#include <linux/err.h>
@@ -66,6 +65,8 @@ static DEFINE_SPINLOCK(kvm_vmid_lock);
static bool vgic_present;
static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled);
static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
{
BUG_ON(preemptible());
@@ -90,11 +91,6 @@ struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void)
return &kvm_arm_running_vcpu;
}
int kvm_arch_hardware_enable(void)
{
return 0;
}
int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu)
{
return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE;
@@ -1033,11 +1029,6 @@ long kvm_arch_vm_ioctl(struct file *filp,
}
}
static void cpu_init_stage2(void *dummy)
{
__cpu_init_stage2();
}
static void cpu_init_hyp_mode(void *dummy)
{
phys_addr_t boot_pgd_ptr;
@@ -1065,43 +1056,87 @@ static void cpu_hyp_reinit(void)
{
if (is_kernel_in_hyp_mode()) {
/*
* cpu_init_stage2() is safe to call even if the PM
* __cpu_init_stage2() is safe to call even if the PM
* event was cancelled before the CPU was reset.
*/
cpu_init_stage2(NULL);
__cpu_init_stage2();
} else {
if (__hyp_get_vectors() == hyp_default_vectors)
cpu_init_hyp_mode(NULL);
}
}
static int hyp_init_cpu_notify(struct notifier_block *self,
unsigned long action, void *cpu)
static void cpu_hyp_reset(void)
{
switch (action) {
case CPU_STARTING:
case CPU_STARTING_FROZEN:
cpu_hyp_reinit();
}
phys_addr_t boot_pgd_ptr;
phys_addr_t phys_idmap_start;
return NOTIFY_OK;
if (!is_kernel_in_hyp_mode()) {
boot_pgd_ptr = kvm_mmu_get_boot_httbr();
phys_idmap_start = kvm_get_idmap_start();
__cpu_reset_hyp_mode(boot_pgd_ptr, phys_idmap_start);
}
}
static struct notifier_block hyp_init_cpu_nb = {
.notifier_call = hyp_init_cpu_notify,
};
static void _kvm_arch_hardware_enable(void *discard)
{
if (!__this_cpu_read(kvm_arm_hardware_enabled)) {
cpu_hyp_reinit();
__this_cpu_write(kvm_arm_hardware_enabled, 1);
}
}
int kvm_arch_hardware_enable(void)
{
_kvm_arch_hardware_enable(NULL);
return 0;
}
static void _kvm_arch_hardware_disable(void *discard)
{
if (__this_cpu_read(kvm_arm_hardware_enabled)) {
cpu_hyp_reset();
__this_cpu_write(kvm_arm_hardware_enabled, 0);
}
}
void kvm_arch_hardware_disable(void)
{
_kvm_arch_hardware_disable(NULL);
}
#ifdef CONFIG_CPU_PM
static int hyp_init_cpu_pm_notifier(struct notifier_block *self,
unsigned long cmd,
void *v)
{
if (cmd == CPU_PM_EXIT) {
cpu_hyp_reinit();
return NOTIFY_OK;
}
/*
* kvm_arm_hardware_enabled is left with its old value over
* PM_ENTER->PM_EXIT. It is used to indicate PM_EXIT should
* re-enable hyp.
*/
switch (cmd) {
case CPU_PM_ENTER:
if (__this_cpu_read(kvm_arm_hardware_enabled))
/*
* don't update kvm_arm_hardware_enabled here
* so that the hardware will be re-enabled
* when we resume. See below.
*/
cpu_hyp_reset();
return NOTIFY_DONE;
return NOTIFY_OK;
case CPU_PM_EXIT:
if (__this_cpu_read(kvm_arm_hardware_enabled))
/* The hardware was enabled before suspend. */
cpu_hyp_reinit();
return NOTIFY_OK;
default:
return NOTIFY_DONE;
}
}
static struct notifier_block hyp_init_cpu_pm_nb = {
@@ -1143,16 +1178,12 @@ static int init_common_resources(void)
static int init_subsystems(void)
{
int err;
int err = 0;
/*
* Register CPU Hotplug notifier
* Enable hardware so that subsystem initialisation can access EL2.
*/
err = register_cpu_notifier(&hyp_init_cpu_nb);
if (err) {
kvm_err("Cannot register KVM init CPU notifier (%d)\n", err);
return err;
}
on_each_cpu(_kvm_arch_hardware_enable, NULL, 1);
/*
* Register CPU lower-power notifier
@@ -1170,9 +1201,10 @@ static int init_subsystems(void)
case -ENODEV:
case -ENXIO:
vgic_present = false;
err = 0;
break;
default:
return err;
goto out;
}
/*
@@ -1180,12 +1212,15 @@ static int init_subsystems(void)
*/
err = kvm_timer_hyp_init();
if (err)
return err;
goto out;
kvm_perf_init();
kvm_coproc_table_init();
return 0;
out:
on_each_cpu(_kvm_arch_hardware_disable, NULL, 1);
return err;
}
static void teardown_hyp_mode(void)
@@ -1198,17 +1233,11 @@ static void teardown_hyp_mode(void)
free_hyp_pgds();
for_each_possible_cpu(cpu)
free_page(per_cpu(kvm_arm_hyp_stack_page, cpu));
unregister_cpu_notifier(&hyp_init_cpu_nb);
hyp_cpu_pm_exit();
}
static int init_vhe_mode(void)
{
/*
* Execute the init code on each CPU.
*/
on_each_cpu(cpu_init_stage2, NULL, 1);
/* set size of VMID supported by CPU */
kvm_vmid_bits = kvm_get_vmid_bits();
kvm_info("%d-bit VMID\n", kvm_vmid_bits);
@@ -1295,11 +1324,6 @@ static int init_hyp_mode(void)
}
}
/*
* Execute the init code on each CPU.
*/
on_each_cpu(cpu_init_hyp_mode, NULL, 1);
#ifndef CONFIG_HOTPLUG_CPU
free_boot_hyp_pgd();
#endif

5
arch/arm/kvm/mmu.c
View File

@@ -1666,6 +1666,11 @@ phys_addr_t kvm_get_idmap_vector(void)
return hyp_idmap_vector;
}
phys_addr_t kvm_get_idmap_start(void)
{
return hyp_idmap_start;
}
int kvm_mmu_init(void)
{
int err;

45
arch/arm64/Kconfig
View File

@@ -11,6 +11,7 @@ config ARM64
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_USE_CMPXCHG_LOCKREF
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_NUMA_BALANCING
select ARCH_WANT_OPTIONAL_GPIOLIB
select ARCH_WANT_COMPAT_IPC_PARSE_VERSION
select ARCH_WANT_FRAME_POINTERS
@@ -58,11 +59,14 @@ config ARM64
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select HAVE_ARM_SMCCC
select HAVE_BPF_JIT
select HAVE_C_RECORDMCOUNT
select HAVE_CC_STACKPROTECTOR
select HAVE_CMPXCHG_DOUBLE
select HAVE_CMPXCHG_LOCAL
select HAVE_CONTEXT_TRACKING
select HAVE_DEBUG_BUGVERBOSE
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_API_DEBUG
@@ -76,6 +80,7 @@ config ARM64
select HAVE_HW_BREAKPOINT if PERF_EVENTS
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP if NUMA
select HAVE_PATA_PLATFORM
select HAVE_PERF_EVENTS
select HAVE_PERF_REGS
@@ -89,15 +94,13 @@ config ARM64
select NO_BOOTMEM
select OF
select OF_EARLY_FLATTREE
select OF_NUMA if NUMA && OF
select OF_RESERVED_MEM
select PERF_USE_VMALLOC
select POWER_RESET
select POWER_SUPPLY
select RTC_LIB
select SPARSE_IRQ
select SYSCTL_EXCEPTION_TRACE
select HAVE_CONTEXT_TRACKING
select HAVE_ARM_SMCCC
help
ARM 64-bit (AArch64) Linux support.
@@ -546,10 +549,35 @@ config HOTPLUG_CPU
Say Y here to experiment with turning CPUs off and on. CPUs
can be controlled through /sys/devices/system/cpu.
# Common NUMA Features
config NUMA
bool "Numa Memory Allocation and Scheduler Support"
depends on SMP
help
Enable NUMA (Non Uniform Memory Access) support.
The kernel will try to allocate memory used by a CPU on the
local memory of the CPU and add some more
NUMA awareness to the kernel.
config NODES_SHIFT
int "Maximum NUMA Nodes (as a power of 2)"
range 1 10
default "2"
depends on NEED_MULTIPLE_NODES
help
Specify the maximum number of NUMA Nodes available on the target
system. Increases memory reserved to accommodate various tables.
config USE_PERCPU_NUMA_NODE_ID
def_bool y
depends on NUMA
source kernel/Kconfig.preempt
source kernel/Kconfig.hz
config ARCH_SUPPORTS_DEBUG_PAGEALLOC
depends on !HIBERNATION
def_bool y
config ARCH_HAS_HOLES_MEMORYMODEL
@@ -578,9 +606,6 @@ config SYS_SUPPORTS_HUGETLBFS
config ARCH_WANT_HUGE_PMD_SHARE
def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
config HAVE_ARCH_TRANSPARENT_HUGEPAGE
def_bool y
config ARCH_HAS_CACHE_LINE_SIZE
def_bool y
@@ -953,6 +978,14 @@ menu "Power management options"
source "kernel/power/Kconfig"
config ARCH_HIBERNATION_POSSIBLE
def_bool y
depends on CPU_PM
config ARCH_HIBERNATION_HEADER
def_bool y
depends on HIBERNATION
config ARCH_SUSPEND_POSSIBLE
def_bool y

2
arch/arm64/Kconfig.debug
View File

@@ -59,7 +59,7 @@ config DEBUG_RODATA
If in doubt, say Y
config DEBUG_ALIGN_RODATA
depends on DEBUG_RODATA && ARM64_4K_PAGES
depends on DEBUG_RODATA
bool "Align linker sections up to SECTION_SIZE"
help
If this option is enabled, sections that may potentially be marked as

133
arch/arm64/include/asm/assembler.h
View File

@@ -1,5 +1,5 @@
/*
* Based on arch/arm/include/asm/assembler.h
* Based on arch/arm/include/asm/assembler.h, arch/arm/mm/proc-macros.S
*
* Copyright (C) 1996-2000 Russell King
* Copyright (C) 2012 ARM Ltd.
@@ -23,21 +23,12 @@
#ifndef __ASM_ASSEMBLER_H
#define __ASM_ASSEMBLER_H
#include <asm/asm-offsets.h>
#include <asm/page.h>
#include <asm/pgtable-hwdef.h>
#include <asm/ptrace.h>
#include <asm/thread_info.h>
/*
* Stack pushing/popping (register pairs only). Equivalent to store decrement
* before, load increment after.
*/
.macro push, xreg1, xreg2
stp \xreg1, \xreg2, [sp, #-16]!
.endm
.macro pop, xreg1, xreg2
ldp \xreg1, \xreg2, [sp], #16
.endm
/*
* Enable and disable interrupts.
*/
@@ -211,6 +202,102 @@ lr .req x30 // link register
add \reg, \reg, \tmp
.endm
/*
* vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
*/
.macro vma_vm_mm, rd, rn
ldr \rd, [\rn, #VMA_VM_MM]
.endm
/*
* mmid - get context id from mm pointer (mm->context.id)
*/
.macro mmid, rd, rn
ldr \rd, [\rn, #MM_CONTEXT_ID]
.endm
/*
* dcache_line_size - get the minimum D-cache line size from the CTR register.
*/
.macro dcache_line_size, reg, tmp
mrs \tmp, ctr_el0 // read CTR
ubfm \tmp, \tmp, #16, #19 // cache line size encoding
mov \reg, #4 // bytes per word
lsl \reg, \reg, \tmp // actual cache line size
.endm
/*
* icache_line_size - get the minimum I-cache line size from the CTR register.
*/
.macro icache_line_size, reg, tmp
mrs \tmp, ctr_el0 // read CTR
and \tmp, \tmp, #0xf // cache line size encoding
mov \reg, #4 // bytes per word
lsl \reg, \reg, \tmp // actual cache line size
.endm
/*
* tcr_set_idmap_t0sz - update TCR.T0SZ so that we can load the ID map
*/
.macro tcr_set_idmap_t0sz, valreg, tmpreg
#ifndef CONFIG_ARM64_VA_BITS_48
ldr_l \tmpreg, idmap_t0sz
bfi \valreg, \tmpreg, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH
#endif
.endm
/*
* Macro to perform a data cache maintenance for the interval
* [kaddr, kaddr + size)
*
* op: operation passed to dc instruction
* domain: domain used in dsb instruciton
* kaddr: starting virtual address of the region
* size: size of the region
* Corrupts: kaddr, size, tmp1, tmp2
*/
.macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2
dcache_line_size \tmp1, \tmp2
add \size, \kaddr, \size
sub \tmp2, \tmp1, #1
bic \kaddr, \kaddr, \tmp2
9998: dc \op, \kaddr
add \kaddr, \kaddr, \tmp1
cmp \kaddr, \size
b.lo 9998b
dsb \domain
.endm
/*
* reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present
*/
.macro reset_pmuserenr_el0, tmpreg
mrs \tmpreg, id_aa64dfr0_el1 // Check ID_AA64DFR0_EL1 PMUVer
sbfx \tmpreg, \tmpreg, #8, #4
cmp \tmpreg, #1 // Skip if no PMU present
b.lt 9000f
msr pmuserenr_el0, xzr // Disable PMU access from EL0
9000:
.endm
/*
* copy_page - copy src to dest using temp registers t1-t8
*/
.macro copy_page dest:req src:req t1:req t2:req t3:req t4:req t5:req t6:req t7:req t8:req
9998: ldp \t1, \t2, [\src]
ldp \t3, \t4, [\src, #16]
ldp \t5, \t6, [\src, #32]
ldp \t7, \t8, [\src, #48]
add \src, \src, #64
stnp \t1, \t2, [\dest]
stnp \t3, \t4, [\dest, #16]
stnp \t5, \t6, [\dest, #32]
stnp \t7, \t8, [\dest, #48]
add \dest, \dest, #64
tst \src, #(PAGE_SIZE - 1)
b.ne 9998b
.endm
/*
* Annotate a function as position independent, i.e., safe to be called before
* the kernel virtual mapping is activated.
@@ -233,4 +320,24 @@ lr .req x30 // link register
.long \sym\()_hi32
.endm
/*
* mov_q - move an immediate constant into a 64-bit register using
* between 2 and 4 movz/movk instructions (depending on the
* magnitude and sign of the operand)
*/
.macro mov_q, reg, val
.if (((\val) >> 31) == 0 || ((\val) >> 31) == 0x1ffffffff)
movz \reg, :abs_g1_s:\val
.else
.if (((\val) >> 47) == 0 || ((\val) >> 47) == 0x1ffff)
movz \reg, :abs_g2_s:\val
.else
movz \reg, :abs_g3:\val
movk \reg, :abs_g2_nc:\val
.endif
movk \reg, :abs_g1_nc:\val
.endif
movk \reg, :abs_g0_nc:\val
.endm
#endif /* __ASM_ASSEMBLER_H */

27
arch/arm64/include/asm/cpufeature.h
View File

@@ -35,8 +35,9 @@
#define ARM64_ALT_PAN_NOT_UAO 10
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
#define ARM64_NCAPS 13
#define ARM64_NCAPS 14
#ifndef __ASSEMBLY__
@@ -77,10 +78,17 @@ struct arm64_ftr_reg {
struct arm64_ftr_bits *ftr_bits;
};
/* scope of capability check */
enum {
SCOPE_SYSTEM,
SCOPE_LOCAL_CPU,
};
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
bool (*matches)(const struct arm64_cpu_capabilities *);
int def_scope; /* default scope */
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
void (*enable)(void *); /* Called on all active CPUs */
union {
struct { /* To be used for erratum handling only */
@@ -101,6 +109,8 @@ struct arm64_cpu_capabilities {
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
bool this_cpu_has_cap(unsigned int cap);
static inline bool cpu_have_feature(unsigned int num)
{
return elf_hwcap & (1UL << num);
@@ -170,12 +180,20 @@ static inline bool id_aa64mmfr0_mixed_endian_el0(u64 mmfr0)
cpuid_feature_extract_unsigned_field(mmfr0, ID_AA64MMFR0_BIGENDEL0_SHIFT) == 0x1;
}
static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
{
u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_EL0_SHIFT);
return val == ID_AA64PFR0_EL0_32BIT_64BIT;
}
void __init setup_cpu_features(void);
void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info);
void check_local_cpu_errata(void);
void verify_local_cpu_errata(void);
void verify_local_cpu_capabilities(void);
u64 read_system_reg(u32 id);
@@ -185,6 +203,11 @@ static inline bool cpu_supports_mixed_endian_el0(void)
return id_aa64mmfr0_mixed_endian_el0(read_cpuid(ID_AA64MMFR0_EL1));
}
static inline bool system_supports_32bit_el0(void)
{
return cpus_have_cap(ARM64_HAS_32BIT_EL0);
}
static inline bool system_supports_mixed_endian_el0(void)
{
return id_aa64mmfr0_mixed_endian_el0(read_system_reg(SYS_ID_AA64MMFR0_EL1));

3
arch/arm64/include/asm/elf.h
View File

@@ -177,7 +177,8 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
/* AArch32 EABI. */
#define EF_ARM_EABI_MASK 0xff000000
#define compat_elf_check_arch(x) (((x)->e_machine == EM_ARM) && \
#define compat_elf_check_arch(x) (system_supports_32bit_el0() && \
((x)->e_machine == EM_ARM) && \
((x)->e_flags & EF_ARM_EABI_MASK))
#define compat_start_thread compat_start_thread

21
arch/arm64/include/asm/kernel-pgtable.h
View File

@@ -19,6 +19,7 @@
#ifndef __ASM_KERNEL_PGTABLE_H
#define __ASM_KERNEL_PGTABLE_H
#include <asm/sparsemem.h>
/*
* The linear mapping and the start of memory are both 2M aligned (per
@@ -86,10 +87,24 @@
* (64k granule), or a multiple that can be mapped using contiguous bits
* in the page tables: 32 * PMD_SIZE (16k granule)
*/
#ifdef CONFIG_ARM64_64K_PAGES
#define ARM64_MEMSTART_ALIGN SZ_512M
#if defined(CONFIG_ARM64_4K_PAGES)
#define ARM64_MEMSTART_SHIFT PUD_SHIFT
#elif defined(CONFIG_ARM64_16K_PAGES)
#define ARM64_MEMSTART_SHIFT (PMD_SHIFT + 5)
#else
#define ARM64_MEMSTART_ALIGN SZ_1G
#define ARM64_MEMSTART_SHIFT PMD_SHIFT
#endif
/*
* sparsemem vmemmap imposes an additional requirement on the alignment of
* memstart_addr, due to the fact that the base of the vmemmap region
* has a direct correspondence, and needs to appear sufficiently aligned
* in the virtual address space.
*/
#if defined(CONFIG_SPARSEMEM_VMEMMAP) && ARM64_MEMSTART_SHIFT < SECTION_SIZE_BITS
#define ARM64_MEMSTART_ALIGN (1UL << SECTION_SIZE_BITS)
#else
#define ARM64_MEMSTART_ALIGN (1UL << ARM64_MEMSTART_SHIFT)
#endif
#endif /* __ASM_KERNEL_PGTABLE_H */

11
arch/arm64/include/asm/kvm_arm.h
View File

@@ -84,17 +84,6 @@
#define HCR_INT_OVERRIDE (HCR_FMO | HCR_IMO)
#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
/* Hyp System Control Register (SCTLR_EL2) bits */
#define SCTLR_EL2_EE (1 << 25)
#define SCTLR_EL2_WXN (1 << 19)
#define SCTLR_EL2_I (1 << 12)
#define SCTLR_EL2_SA (1 << 3)
#define SCTLR_EL2_C (1 << 2)
#define SCTLR_EL2_A (1 << 1)
#define SCTLR_EL2_M 1
#define SCTLR_EL2_FLAGS (SCTLR_EL2_M | SCTLR_EL2_A | SCTLR_EL2_C | \
SCTLR_EL2_SA | SCTLR_EL2_I)
/* TCR_EL2 Registers bits */
#define TCR_EL2_RES1 ((1 << 31) | (1 << 23))
#define TCR_EL2_TBI (1 << 20)

3
arch/arm64/include/asm/kvm_asm.h
View File

@@ -22,6 +22,8 @@
#define ARM_EXCEPTION_IRQ 0
#define ARM_EXCEPTION_TRAP 1
/* The hyp-stub will return this for any kvm_call_hyp() call */
#define ARM_EXCEPTION_HYP_GONE 2
#define KVM_ARM64_DEBUG_DIRTY_SHIFT 0
#define KVM_ARM64_DEBUG_DIRTY (1 << KVM_ARM64_DEBUG_DIRTY_SHIFT)
@@ -40,6 +42,7 @@ struct kvm_vcpu;
extern char __kvm_hyp_init[];
extern char __kvm_hyp_init_end[];
extern char __kvm_hyp_reset[];
extern char __kvm_hyp_vector[];

14
arch/arm64/include/asm/kvm_host.h
View File

@@ -46,6 +46,8 @@
int __attribute_const__ kvm_target_cpu(void);
int kvm_reset_vcpu(struct kvm_vcpu *vcpu);
int kvm_arch_dev_ioctl_check_extension(long ext);
unsigned long kvm_hyp_reset_entry(void);
void __extended_idmap_trampoline(phys_addr_t boot_pgd, phys_addr_t idmap_start);
struct kvm_arch {
/* The VMID generation used for the virt. memory system */
@@ -352,7 +354,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr,
hyp_stack_ptr, vector_ptr);
}
static inline void kvm_arch_hardware_disable(void) {}
static inline void __cpu_reset_hyp_mode(phys_addr_t boot_pgd_ptr,
phys_addr_t phys_idmap_start)
{
/*
* Call reset code, and switch back to stub hyp vectors.
* Uses __kvm_call_hyp() to avoid kaslr's kvm_ksym_ref() translation.
*/
__kvm_call_hyp((void *)kvm_hyp_reset_entry(),
boot_pgd_ptr, phys_idmap_start);
}
static inline void kvm_arch_hardware_unsetup(void) {}
static inline void kvm_arch_sync_events(struct kvm *kvm) {}
static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}

1
arch/arm64/include/asm/kvm_mmu.h
View File

@@ -109,6 +109,7 @@ void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
phys_addr_t kvm_mmu_get_httbr(void);
phys_addr_t kvm_mmu_get_boot_httbr(void);
phys_addr_t kvm_get_idmap_vector(void);
phys_addr_t kvm_get_idmap_start(void);
int kvm_mmu_init(void);
void kvm_clear_hyp_idmap(void);

33
arch/arm64/include/asm/memory.h
View File

@@ -39,6 +39,21 @@
*/
#define PCI_IO_SIZE SZ_16M
/*
* Log2 of the upper bound of the size of a struct page. Used for sizing
* the vmemmap region only, does not affect actual memory footprint.
* We don't use sizeof(struct page) directly since taking its size here
* requires its definition to be available at this point in the inclusion
* chain, and it may not be a power of 2 in the first place.
*/
#define STRUCT_PAGE_MAX_SHIFT 6
/*
* VMEMMAP_SIZE - allows the whole linear region to be covered by
* a struct page array
*/
#define VMEMMAP_SIZE (UL(1) << (VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT))
/*
* PAGE_OFFSET - the virtual address of the start of the kernel image (top
* (VA_BITS - 1))
@@ -54,7 +69,8 @@
#define MODULES_END (MODULES_VADDR + MODULES_VSIZE)
#define MODULES_VADDR (VA_START + KASAN_SHADOW_SIZE)
#define MODULES_VSIZE (SZ_128M)
#define PCI_IO_END (PAGE_OFFSET - SZ_2M)
#define VMEMMAP_START (PAGE_OFFSET - VMEMMAP_SIZE)
#define PCI_IO_END (VMEMMAP_START - SZ_2M)
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
#define FIXADDR_TOP (PCI_IO_START - SZ_2M)
#define TASK_SIZE_64 (UL(1) << VA_BITS)
@@ -71,6 +87,9 @@
#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4))
#define KERNEL_START _text
#define KERNEL_END _end
/*
* The size of the KASAN shadow region. This should be 1/8th of the
* size of the entire kernel virtual address space.
@@ -192,9 +211,19 @@ static inline void *phys_to_virt(phys_addr_t x)
*/
#define ARCH_PFN_OFFSET ((unsigned long)PHYS_PFN_OFFSET)
#ifndef CONFIG_SPARSEMEM_VMEMMAP
#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
#define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
#define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
#else
#define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page))
#define __page_to_voff(kaddr) (((u64)(page) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page))
#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET))
#define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START))
#define virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \
+ PHYS_OFFSET) >> PAGE_SHIFT)
#endif
#endif
#include <asm-generic/memory_model.h>

1
arch/arm64/include/asm/mmu.h
View File

@@ -29,6 +29,7 @@ typedef struct {
#define ASID(mm) ((mm)->context.id.counter & 0xffff)
extern void paging_init(void);
extern void bootmem_init(void);
extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt);
extern void init_mem_pgprot(void);
extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys,

12
arch/arm64/include/asm/mmzone.h Normal file
View File

@@ -0,0 +1,12 @@
#ifndef __ASM_MMZONE_H
#define __ASM_MMZONE_H
#ifdef CONFIG_NUMA
#include <asm/numa.h>
extern struct pglist_data *node_data[];
#define NODE_DATA(nid) (node_data[(nid)])
#endif /* CONFIG_NUMA */
#endif /* __ASM_MMZONE_H */

Some files were not shown because too many files have changed in this diff Show More