f30a578d76
powerpc/code-patching: Move code patching selftests in its own file
...
Code patching selftests are half of code-patching.c.
As they are guarded by CONFIG_CODE_PATCHING_SELFTESTS,
they'd be better in their own file.
Also add a missing __init for instr_is_branch_to_addr()
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/c0c30504f04eb546a48ff77127a8bccd12a3d809.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:36:58 +11:00
31acc59956
powerpc/code-patching: Move instr_is_branch_{i/b}form() in code-patching.h
...
To enable moving selftests in their own C file in following patch,
move instr_is_branch_iform() and instr_is_branch_bform()
to code-patching.h
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/fca0f3b191211b3681020885a611bf73eef20563.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:36:58 +11:00
29562a9da2
powerpc/code-patching: Move patch_exception() outside code-patching.c
...
patch_exception() is dedicated to book3e/64 is nothing more than
a normal use of patch_branch(), so move it into a place dedicated
to book3e/64.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/0968622b98b1fb51838c35b844c42ad6609de62e.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:36:55 +11:00
ff14a9c09f
powerpc/code-patching: Use test_trampoline for prefixed patch test
...
Use the dedicated test_trampoline function for testing prefixed
patching like other tests and remove the hand coded assembly stuff.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/a450ef3f8653f75e1bd9aaf7a3889d379752f33b.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:25 +11:00
d5937db114
powerpc/code-patching: Fix patch_branch() return on out-of-range failure
...
Do not silentely ignore a failure of create_branch() in
patch_branch(). Return -ERANGE.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/8540cb64b1f06710eaf41e3835c7ba3e21fa2b05.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
6b21af7449
powerpc/code-patching: Reorganise do_patch_instruction() to ease error handling
...
Split do_patch_instruction() in two functions, the caller doing the
spin locking and the callee doing everything else.
And remove a few unnecessary initialisations and intermediate
variables.
This allows the callee to return from anywhere in the function.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/dbc85980a0d2a935731b272e8907e8bb1d8fc8c5.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
a3483c3dd1
powerpc/code-patching: Fix unmap_patch_area() error handling
...
pXd_offset() doesn't return NULL. When the base is NULL, it
still adds the offset.
Use pXd_none() to check validity instead. It also improves
performance by folding out none existing levels as pXd_none()
always returns 0 in that case.
Such an error is unexpected, use WARN_ON() so that the caller
doesn't have to worry about it, and drop the returned value.
And now that unmap_patch_area() doesn't return error, we can
take into account the error returned by __patch_instruction().
While at it, remove the 'inline' property which is useless.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/299804b117fae35c786c827536c91f25352e279b.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
285672f993
powerpc/code-patching: Fix error handling in do_patch_instruction()
...
Use real errors instead of using -1 as error, so that errors
returned by callees can be used towards callers.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/85259d894069e47f915ea580b169e1adbeec7a61.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
af5304a750
powerpc/code-patching: Remove init_mem_is_free
...
A new state has been added by commit d2635f2012christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/ad8c3ccb39c8edaa89fd3eda1cc7218baea1cde5.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
edecd2d6d6
powerpc/code-patching: Remove pr_debug()/pr_devel() messages and fix check()
...
code-patching has been working for years now, time has come to
remove debugging messages.
Change useful message to KERN_INFO and remove other ones.
Also add KERN_ERR to check() macro and change it into a do/while
to make checkpatch happy.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/3ff9823c0a812a8a145d979a9600a6d4591b80ee.1638446239.git.christophe.leroy@csgroup.eu
2021-12-23 22:35:24 +11:00
ce0c6be9c6
powerpc/lib: Add __init attribute to eligible functions
...
Some functions defined in 'arch/powerpc/lib' are deserving of an `__init`
macro attribute. These functions are only called by other initialization
functions and therefore should inherit the attribute.
Also, change function declarations in header files to include `__init`.
Signed-off-by: Nick Child <nick.child@ibm.com >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/20211216220035.605465-3-nick.child@ibm.com
2021-12-23 22:33:10 +11:00
c545b9f040
powerpc/inst: Define ppc_inst_t
...
In order to stop using 'struct ppc_inst' on PPC32,
define a ppc_inst_t typedef.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/fe5baa2c66fea9db05a8b300b3e8d2880a42596c.1638208156.git.christophe.leroy@csgroup.eu
2021-12-09 22:41:21 +11:00
af3fdce4ab
Revert "powerpc/code-patching: Improve verification of patchability"
...
This reverts commit 8b8a8f0ab3https://lore.kernel.org/linuxppc-dev/3668743C-09DF-4673-B15C-2FFE2A57F7D7@linux.vnet.ibm.com/
2: https://lore.kernel.org/linuxppc-dev/20211126161747.1f7795b0@canb.auug.org.au/
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
2021-11-29 17:41:52 +11:00
8b8a8f0ab3
powerpc/code-patching: Improve verification of patchability
...
Today, patch_instruction() assumes that it is called exclusively on
valid addresses, and only checks that it is not called on an init
address after init section has been freed.
Improve verification by calling kernel_text_address() instead.
kernel_text_address() already includes a verification of
initmem release.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/bc683d499a411730504b132a924de0ccc2ef1f79.1636971137.git.christophe.leroy@csgroup.eu
2021-11-25 11:25:32 +11:00
4549c3ea31
powerpc/lib: Add helper to check if offset is within conditional branch range
...
Add a helper to check if a given offset is within the branch range for a
powerpc conditional branch instruction, and update some sites to use the
new helper.
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com >
Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Acked-by: Song Liu <songliubraving@fb.com >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/442b69a34ced32ca346a0d9a855f3f6cfdbbbd41.1633464148.git.naveen.n.rao@linux.vnet.ibm.com
2021-10-07 18:52:33 +11:00
71a5b3db9f
powerpc/lib/code-patching: Set up Strict RWX patching earlier
...
setup_text_poke_area() is a late init call so it runs before
mark_rodata_ro() and after the init calls. This lets all the init code
patching simply write to their locations. In the future, kprobes is
going to allocate its instruction pages RO which means they will need
setup_text__poke_area() to have been already called for their code
patching. However, init_kprobes() (which allocates and patches some
instruction pages) is an early init call so it happens before
setup_text__poke_area().
start_kernel() calls poking_init() before any of the init calls. On
powerpc, poking_init() is currently a nop. setup_text_poke_area() relies
on kernel virtual memory, cpu hotplug and per_cpu_areas being setup.
setup_per_cpu_areas(), boot_cpu_hotplug_init() and mm_init() are called
before poking_init().
Turn setup_text_poke_area() into poking_init().
Signed-off-by: Jordan Niethe <jniethe5@gmail.com >
Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Reviewed-by: Russell Currey <ruscur@russell.cc >
[mpe: Fold in missing prototype for poking_init() from lkp]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/20210609013431.9805-3-jniethe5@gmail.com
2021-06-21 21:13:20 +10:00
69d4d6e5fd
powerpc: Don't use 'struct ppc_inst' to reference instruction location
...
'struct ppc_inst' is an internal representation of an instruction, but
in-memory instructions are and will remain a table of 'u32' forever.
Replace all 'struct ppc_inst *' used for locating an instruction in
memory by 'u32 *'. This removes a lot of undue casts to 'struct
ppc_inst *'.
It also helps locating ab-use of 'struct ppc_inst' dereference.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
[mpe: Fix ppc_inst_next(), use u32 instead of unsigned int]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/7062722b087228e42cbd896e39bfdf526d6a340a.1621516826.git.christophe.leroy@csgroup.eu
2021-06-17 00:09:00 +10:00
e90a21ea80
powerpc/lib/code-patching: Don't use struct 'ppc_inst' for runnable code in tests.
...
'struct ppc_inst' is meant to represent an instruction internally, it
is not meant to dereference code in memory.
For testing code patching, use patch_instruction() to properly
write into memory the code to be tested.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/d8425fb42a4adebc35b7509f121817eeb02fac31.1621516826.git.christophe.leroy@csgroup.eu
2021-06-17 00:07:51 +10:00
6c0d181daa
powerpc/lib/code-patching: Make instr_is_branch_to_addr() static
...
instr_is_branch_to_addr() is only used in code-patching.c
Make it static.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/5f6b9c8c83170ed310953eac2f5b14539bfc964a.1621516826.git.christophe.leroy@csgroup.eu
2021-06-16 23:35:57 +10:00
18c85964b1
powerpc: Do not dereference code as 'struct ppc_inst' (uprobe, code-patching, feature-fixups)
...
'struct ppc_inst' is an internal structure to represent an instruction,
it is not directly the representation of that instruction in text code.
It is not meant to map and dereference code.
Dereferencing code directly through 'struct ppc_inst' has two main issues:
- On powerpc, structs are expected to be 8 bytes aligned while code is
spread every 4 byte.
- Should a non prefixed instruction lie at the end of the page and the
following page not be mapped, it would generate a page fault.
In-memory code must be accessed with ppc_inst_read().
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/c9a1201dd0a66b4a0f91f0fb46d9385cbf030feb.1621516826.git.christophe.leroy@csgroup.eu
2021-06-16 23:35:57 +10:00
693557ebf4
powerpc/inst: ppc_inst_as_u64() becomes ppc_inst_as_ulong()
...
In order to simplify use on PPC32, change ppc_inst_as_u64()
into ppc_inst_as_ulong() that returns the 32 bits instruction
on PPC32.
Will be used when porting OPTPROBES to PPC32.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/22cadf29620664b600b82026d2a72b8b23351777.1618927318.git.christophe.leroy@csgroup.eu
2021-04-21 22:52:32 +10:00
e63ceebdad
powerpc/lib: Don't use __put_user_asm_goto() outside of uaccess.h
...
__put_user_asm_goto() is internal to uaccess.h
Use __put_kernel_nofault() instead. The generated code is identical.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/3e32c4f0361933909368b68f5ee569e5de661c1b.1615398498.git.christophe.leroy@csgroup.eu
2021-03-26 23:19:42 +11:00
e64ac41ab0
powerpc/uaccess: Switch __patch_instruction() to __put_user_asm_goto()
...
__patch_instruction() is the only user of __put_user_asm() outside
of asm/uaccess.h
Switch to the new __put_user_asm_goto() to enable retirement of
__put_user_asm() in a later patch.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/b9745b122f4a9ae72cef445c61320022ab8b77b7.1599216721.git.christophe.leroy@csgroup.eu
2020-09-15 22:13:32 +10:00
ccc8fcf72a
powerpc/lib: Prepare code-patching for modules allocated outside vmalloc space
...
Use is_vmalloc_or_module_addr() instead of is_vmalloc_addr()
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu >
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au >
Link: https://lore.kernel.org/r/7d884db0e5a6f521331639d8c0f13e520d5a4fef.1593428200.git.christophe.leroy@csgroup.eu
2020-07-27 00:01:30 +10:00
e31cf2f4ca
mm: don't include asm/pgtable.h if linux/mm.h is already included
...
Patch series "mm: consolidate definitions of page table accessors", v2.
The low level page table accessors (pXY_index(), pXY_offset()) are
duplicated across all architectures and sometimes more than once. For
instance, we have 31 definition of pgd_offset() for 25 supported
architectures.
Most of these definitions are actually identical and typically it boils
down to, e.g.
static inline unsigned long pmd_index(unsigned long address)
{
return (address >> PMD_SHIFT) & (PTRS_PER_PMD - 1);
}
static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
{
return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(address);
}
These definitions can be shared among 90% of the arches provided
XYZ_SHIFT, PTRS_PER_XYZ and xyz_page_vaddr() are defined.
For architectures that really need a custom version there is always
possibility to override the generic version with the usual ifdefs magic.
These patches introduce include/linux/pgtable.h that replaces
include/asm-generic/pgtable.h and add the definitions of the page table
accessors to the new header.
This patch (of 12):
The linux/mm.h header includes <asm/pgtable.h> to allow inlining of the
functions involving page table manipulations, e.g. pte_alloc() and
pmd_alloc(). So, there is no point to explicitly include <asm/pgtable.h>
in the files that include <linux/mm.h>.
The include statements in such cases are remove with a simple loop:
for f in $(git grep -l "include <linux/mm.h>") ; do
sed -i -e '/include <asm\/pgtable.h>/ d' $f
done
Signed-off-by: Mike Rapoport <rppt@linux.ibm.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Cc: Arnd Bergmann <arnd@arndb.de >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Brian Cain <bcain@codeaurora.org >
Cc: Catalin Marinas <catalin.marinas@arm.com >
Cc: Chris Zankel <chris@zankel.net >
Cc: "David S. Miller" <davem@davemloft.net >
Cc: Geert Uytterhoeven <geert@linux-m68k.org >
Cc: Greentime Hu <green.hu@gmail.com >
Cc: Greg Ungerer <gerg@linux-m68k.org >
Cc: Guan Xuetao <gxt@pku.edu.cn >
Cc: Guo Ren <guoren@kernel.org >
Cc: Heiko Carstens <heiko.carstens@de.ibm.com >
Cc: Helge Deller <deller@gmx.de >
Cc: Ingo Molnar <mingo@redhat.com >
Cc: Ley Foon Tan <ley.foon.tan@intel.com >
Cc: Mark Salter <msalter@redhat.com >
Cc: Matthew Wilcox <willy@infradead.org >
Cc: Matt Turner <mattst88@gmail.com >
Cc: Max Filippov <jcmvbkbc@gmail.com >
Cc: Michael Ellerman <mpe@ellerman.id.au >
Cc: Michal Simek <monstr@monstr.eu >
Cc: Mike Rapoport <rppt@kernel.org >
Cc: Nick Hu <nickhu@andestech.com >
Cc: Paul Walmsley <paul.walmsley@sifive.com >
Cc: Richard Weinberger <richard@nod.at >
Cc: Rich Felker <dalias@libc.org >
Cc: Russell King <linux@armlinux.org.uk >
Cc: Stafford Horne <shorne@gmail.com >
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: Tony Luck <tony.luck@intel.com >
Cc: Vincent Chen <deanbo422@gmail.com >
Cc: Vineet Gupta <vgupta@synopsys.com >
Cc: Will Deacon <will@kernel.org >
Cc: Yoshinori Sato <ysato@users.sourceforge.jp >
Link: http://lkml.kernel.org/r/20200514170327.31389-1-rppt@kernel.org
Link: http://lkml.kernel.org/r/20200514170327.31389-2-rppt@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2020-06-09 09:39:13 -07:00
Christophe Leroy
Nick Child
Michael Ellerman
Naveen N. Rao
Jordan Niethe
Mike Rapoport