You've already forked linux-apfs
mirror of
https://github.com/linux-apfs/linux-apfs.git
synced 2026-05-01 15:00:59 -07:00
Suresh Siddha
304bceda6a
Fundamental model of the current Linux kernel is to lazily init and restore FPU instead of restoring the task state during context switch. This changes that fundamental lazy model to the non-lazy model for the processors supporting xsave feature. Reasons driving this model change are: i. Newer processors support optimized state save/restore using xsaveopt and xrstor by tracking the INIT state and MODIFIED state during context-switch. This is faster than modifying the cr0.TS bit which has serializing semantics. ii. Newer glibc versions use SSE for some of the optimized copy/clear routines. With certain workloads (like boot, kernel-compilation etc), application completes its work with in the first 5 task switches, thus taking upto 5 #DNA traps with the kernel not getting a chance to apply the above mentioned pre-load heuristic. iii. Some xstate features (like AMD's LWP feature) don't honor the cr0.TS bit and thus will not work correctly in the presence of lazy restore. Non-lazy state restore is needed for enabling such features. Some data on a two socket SNB system: * Saved 20K DNA exceptions during boot on a two socket SNB system. * Saved 50K DNA exceptions during kernel-compilation workload. * Improved throughput of the AVX based checksumming function inside the kernel by ~15% as xsave/xrstor is faster than the serializing clts/stts pair. Also now kernel_fpu_begin/end() relies on the patched alternative instructions. So move check_fpu() which uses the kernel_fpu_begin/end() after alternative_instructions(). Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com> Link: http://lkml.kernel.org/r/1345842782-24175-7-git-send-email-suresh.b.siddha@intel.com Merge 32-bit boot fix from, Link: http://lkml.kernel.org/r/1347300665-6209-4-git-send-email-suresh.b.siddha@intel.com Cc: Jim Kukunas <james.t.kukunas@linux.intel.com> Cc: NeilBrown <neilb@suse.de> Cc: Avi Kivity <avi@redhat.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
81 lines
2.0 KiB
C
81 lines
2.0 KiB
C
/*
|
|
* Copyright (C) 1994 Linus Torvalds
|
|
*
|
|
* Pentium III FXSR, SSE support
|
|
* General FPU state handling cleanups
|
|
* Gareth Hughes <gareth@valinux.com>, May 2000
|
|
* x86-64 work by Andi Kleen 2002
|
|
*/
|
|
|
|
#ifndef _ASM_X86_I387_H
|
|
#define _ASM_X86_I387_H
|
|
|
|
#ifndef __ASSEMBLY__
|
|
|
|
#include <linux/sched.h>
|
|
#include <linux/hardirq.h>
|
|
|
|
struct pt_regs;
|
|
struct user_i387_struct;
|
|
|
|
extern int init_fpu(struct task_struct *child);
|
|
extern void fpu_finit(struct fpu *fpu);
|
|
extern int dump_fpu(struct pt_regs *, struct user_i387_struct *);
|
|
extern void math_state_restore(void);
|
|
|
|
extern bool irq_fpu_usable(void);
|
|
extern void kernel_fpu_begin(void);
|
|
extern void kernel_fpu_end(void);
|
|
|
|
/*
|
|
* Some instructions like VIA's padlock instructions generate a spurious
|
|
* DNA fault but don't modify SSE registers. And these instructions
|
|
* get used from interrupt context as well. To prevent these kernel instructions
|
|
* in interrupt context interacting wrongly with other user/kernel fpu usage, we
|
|
* should use them only in the context of irq_ts_save/restore()
|
|
*/
|
|
static inline int irq_ts_save(void)
|
|
{
|
|
/*
|
|
* If in process context and not atomic, we can take a spurious DNA fault.
|
|
* Otherwise, doing clts() in process context requires disabling preemption
|
|
* or some heavy lifting like kernel_fpu_begin()
|
|
*/
|
|
if (!in_atomic())
|
|
return 0;
|
|
|
|
if (read_cr0() & X86_CR0_TS) {
|
|
clts();
|
|
return 1;
|
|
}
|
|
|
|
return 0;
|
|
}
|
|
|
|
static inline void irq_ts_restore(int TS_state)
|
|
{
|
|
if (TS_state)
|
|
stts();
|
|
}
|
|
|
|
/*
|
|
* The question "does this thread have fpu access?"
|
|
* is slightly racy, since preemption could come in
|
|
* and revoke it immediately after the test.
|
|
*
|
|
* However, even in that very unlikely scenario,
|
|
* we can just assume we have FPU access - typically
|
|
* to save the FP state - we'll just take a #NM
|
|
* fault and get the FPU access back.
|
|
*/
|
|
static inline int user_has_fpu(void)
|
|
{
|
|
return current->thread.fpu.has_fpu;
|
|
}
|
|
|
|
extern void unlazy_fpu(struct task_struct *tsk);
|
|
|
|
#endif /* __ASSEMBLY__ */
|
|
|
|
#endif /* _ASM_X86_I387_H */
|