Ingo Molnar
df6b35f409
x86/fpu: Rename i387.h to fpu/api.h
...
We already have fpu/types.h, move i387.h to fpu/api.h.
The file name has become a misnomer anyway: it offers generic FPU APIs,
but is not limited to i387 functionality.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:30 +02:00
Ingo Molnar
e11267c13f
x86/fpu: Clean up fpu__clear() a bit
...
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:30 +02:00
Ingo Molnar
2e8a310266
x86/fpu: Rename fpu__flush_thread() to fpu__clear()
...
The primary purpose of this function is to clear the current task's
FPU before an exec(), to not leak information from the previous task,
and to allow the new task to start with freshly initialized FPU
registers.
Rename the function to reflect this primary purpose.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:29 +02:00
Ingo Molnar
cc08d54599
x86/fpu: Use 'struct fpu' in fpu__unlazy_stopped()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:29 +02:00
Ingo Molnar
db2b1d3ad1
x86/fpu: Use 'struct fpu' in fpstate_alloc_init()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:29 +02:00
Ingo Molnar
c69e098b1f
x86/fpu: Use 'struct fpu' in fpu__copy()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:29 +02:00
Ingo Molnar
f9bc977fe7
x86/fpu: Use 'struct fpu' in fpu_copy()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:28 +02:00
Ingo Molnar
0c070595ce
x86/fpu: Use 'struct fpu' in fpu__save()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:28 +02:00
Ingo Molnar
a4d8fc2e06
x86/fpu: Use 'struct fpu' in __fpu_save()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:28 +02:00
Ingo Molnar
2d75bcf314
x86/fpu: Move __save_fpu() into fpu/core.c
...
This helper function is only used in fpu/core.c, move it there.
This slightly speeds up compilation.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:27 +02:00
Ingo Molnar
384a23f939
x86/fpu: Use 'struct fpu' in switch_fpu_finish()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:27 +02:00
Ingo Molnar
cb8818b6ac
x86/fpu: Use 'struct fpu' in switch_fpu_prepare()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:27 +02:00
Ingo Molnar
af2d94fddc
x86/fpu: Use 'struct fpu' in fpu_reset_state()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:26 +02:00
Ingo Molnar
11f2d50b10
x86/fpu: Use 'struct fpu' in restore_fpu_checking()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:26 +02:00
Ingo Molnar
66ddc2cb0f
x86/fpu: Use 'struct fpu' in fpu_lazy_restore()
...
Also rename it to fpu_want_lazy_restore(), to better indicate that
this function just tests whether we can do a lazy restore. (The old
name suggested that it was doing the lazy restore, which is not
the case.)
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:26 +02:00
Ingo Molnar
eb6a3251bf
x86/fpu: Remove task_disable_lazy_fpu_restore()
...
Replace task_disable_lazy_fpu_restore() with easier to read
open-coded uses: we already update the fpu->last_cpu field
explicitly in other cases.
(This also removes yet another task_struct using FPU method.)
Better explain the fpu::last_cpu field in the structure definition.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:26 +02:00
Ingo Molnar
ca6787ba0f
x86/fpu: Remove 'struct task_struct' usage from drop_fpu()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:25 +02:00
Ingo Molnar
c5bedc6847
x86/fpu: Get rid of PF_USED_MATH usage, convert it to fpu->fpstate_active
...
Introduce a simple fpu->fpstate_active flag in the fpu context data structure
and use that instead of PF_USED_MATH in task->flags.
Testing for this flag byte should be slightly more efficient than
testing a bit in a bitmask, but the main advantage is that most
FPU functions can now be performed on a 'struct fpu' alone, they
don't need access to 'struct task_struct' anymore.
There's a slight linecount increase, mostly due to the 'fpu' local
variables and due to extra comments. The local variables will go away
once we move most of the FPU methods to pure 'struct fpu' parameters.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:25 +02:00
Ingo Molnar
af7f8721f1
x86/fpu: Document fpu__unlazy_stopped()
...
Explain its usage and also document a TODO item.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:25 +02:00
Ingo Molnar
4c1384100e
x86/fpu: Open code PF_USED_MATH usages
...
PF_USED_MATH is used directly, but also in a handful of helper inlines.
To ease the elimination of PF_USED_MATH, convert all inline helpers
to open-coded PF_USED_MATH usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:24 +02:00
Ingo Molnar
4540d3faa7
x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_begin()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:24 +02:00
Ingo Molnar
35191e3f07
x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_end()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:24 +02:00
Ingo Molnar
c0311f63e3
x86/fpu: Remove 'struct task_struct' usage from __thread_set_has_fpu()
...
Migrate this function to pure 'struct fpu' usage.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:23 +02:00
Ingo Molnar
36b544dcd3
x86/fpu: Change fpu_owner_task to fpu_fpregs_owner_ctx
...
Track the FPU owner context instead of the owner task: this change,
together with other changes, will allow in subsequent patches the
elimination of 'struct task_struct' usage in various FPU code:
we'll be able to use 'struct fpu' only.
There's no change in code size:
text data bss dec hex filename
13066467 2545248 1626112 17237827 1070743 vmlinux.before
13066467 2545248 1626112 17237827 1070743 vmlinux.after
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:23 +02:00
Ingo Molnar
b0c050c5ba
x86/fpu: Move 'PER_CPU(fpu_owner_task)' to fpu/core.c
...
Move it closer to other per-cpu FPU data structures.
This also unifies the 32-bit and 64-bit code.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:23 +02:00